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ABSTRACT 

This report describes a study conducted to provide 
information that could be used as a basis for program decision malcing 
when adult basic education (ABE) program personnel in Washington 
purchase an integrated computerized learning system. During the 
project, the following activities were carried out: (1) programs and 
systems in use in ABE programs in the state of Washington during the 
1989-90 school year were identified; (2) an advisory committee was 
formed; (3) evaluation criteria for use in looking at large 
computer-based instructional systems were develr>ped; (4) five systems 
currently in use m Washington that have high consumer interest were 
identified; (5) an in-depth, on-site examination of these five 
systems was conducted; and (6) a final report was prepared with 
specific information about each system. Systems evaluated and 
described in this report are the CCC Instructional System (Computer 
Curriculum Corporation, Palo Alto, California); CCP System 
(Comprehensive Competencies Prograun, U.S. BASICS, Alexandria, 
Virginia); ClassWorks System (Computer Networking Specialists, Walla 
Walla, Washington); PLATO System (Control Data, Rolling Meadows, 
Illinois); and WICAT System (WICAT Systems, Orem, Utah). A matrix 
compares characteristics of the five systems. Survey results and the 
evaluation instrument and checklist are appended. (KC) 
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Section 1: FINAL BEPORT SUMMARY 



Introductloo 

During the last several years, there has been a proliferation of large and expensive computer-based instruc- 
tional systems aimed at the population of adults who require remediation in basic skills. These systems often 
range in price from $50,000 to $125,000. The companies which produce these programs base their advertis- 
ing on claims that these systems are efficient and effective, allow dramitic grade level gains with limited 
instructional hours, and minimize the need for professional staff. Some of these systems are, in fact, student 
management systems, while others are a collection of drill and practice exercises. Staffs of ABE programs are 
faced with decisions on what equipment and software are appropriate for purchase. They must know what 
available options would best meet their program's needs and would be the best expenditure of limited dollars 
available. 

Recognizing that much of the information which comes to a program is gained from the salesperson repre- 
senting any given system, this Specia! Project was designed to provide basic information which could be used 
as a basis for program decision-making when ABE programs decide to purchase an integrated learning 
system. The project set the following goals: 

1. To identify programs and systems in use in ABE programs in the state of Washington during the 
1989-1990 school year 

2. To form an advisory committee of interested individuals who have practical experience with the^e 
various systems. 

3. To establish a set of evaluation criteria suitable for use in looking at large computer-based instruc 
lional systems. To review and finalize these criteria with the advisory committee. 

4. To identify five systems currently in use in Washington which have high consumer interest. 

5. To conduct an in-depth, on-site examination of each of these five systems. 

6. To prepare a final report with the examination instrument and specific information on each system 
evaluated. 

The Advliory Commlllec 

The following persons agreed to serve as an informal Advisory Committee: 

Susan Fish, Director of Adult Basic Education at Big Bend Community College 
Bill Guise, Director of Federal Programs for the Highline School District 
May Haley, CCP Lab Coordinator at Skagit Valley College 

Richard Johnson, Learning Center Director at Washington Institute of Applied Technology 
Kathy Kieffer, Learning Lab Coordinator at Clover Park VTI 
Daren vom Steeg, ABE Director at Clallam Bay Correctional Facility 

The Preliminaiy Survey 

All Adult Basic Education programs in Washmgton, Oregon, and British Columbia were surveyed during 
October, 1989, in order to identify which systems were currently in use m those states. This list can seive as 
a resource to other programs which may wish to do on-site visits prior to purchasing new equipment. The 
survey results are contained in Appendix 1, p. 35. 

The Evalaation Criteria Inetmnienl 

With the help of the Advisory Committee, an extensive evaluation document was developed, reviewed, and 
modified by all advisory committee members. This tool was subsequently used in the on-site vis:cs to specific 
programs. It is an appropriate instrument for other programs to use for self-diagnosis when determining 
what systems may be appropriate for their purchase. The Evaluation instrument is included as Appendix 2, p. 
37. Also developed was a checklist form of the instrument, which lists items to be considered \r\ making any 
program determination. This short form is included as Appendix 3, p. 44. 

Selection of the Syttenu for In-depth Exnmination 

The project staff consulted with the Advisory Committee to determine which five systems were most appro- 
priate for the in-depth study comprising the major part of the project. The systems selected were: 




Integrated Uaming Systems 
Page2 



CCC, Computer Curriculum Corporation 
CCP, Comprehensive Competencies Program 
ClassWorlo, Computer Networking Specialists 
PLATO, The Roach Organization 
WICAT.WICAT Systems 

Individual sites were also identified. At the recommendation of the committee, however, it was decided to let 
the publisher's representative pick the site for the in-depth visit, thereby insuring that a model site was 
visited. 



sittvisiu 

Each site was visited for at least one full day. Staff and students were interviewed, and the examination in- 
strument was completed. The specific reports on each program observed make up Section Four, starting on 
page 1 1 of this Final Report 

Gtaeral Imms for CoMidentloa 

After all site visits were concluded, a series of General Issues for Consideration were identified. These issues, 
discussed in Section Two, pp. 4-6, should be considered by any program which is in the process of selecting 
any Integrated Learning System for purchase or lease. 

Sptdfk ItNfS for Coasldtntloa 

Besides the general issues which should be resolved in deciding to move to a large, computer-based instruc- 
tional system, there are a series of specific issues which shou*^ be analyzed in relation to each product under 
consideration. These issues are discussed in Section Three, po i-lO. It is recommer.ded that any program 
reach clear decisions on the importance of each of these factors when seiecting any program. 

Slsf tttm Rtporti 

Section Four, starting on p. 11 of this report, includes the detailed description and analysis of each of the five 
systems studied. Strengths and weaknesses, m the opinion of the researcher, are discussed. 

Sammary Malrli 

In addition to the in-dep;h discussion of each system, a Summary Matrix (Section 5) was developed which 
allows staff to see the key points for consideration in each of the five systems. Project staff urge, however, 
that any program considering purchase of an Integrated Learning System read the detailed discussion 
carefully before selecting a system. 

Find Report Condadonf 

This Final Report for the Project provides information on the process under which the project v;as con- 
ducted, as well as the detailed content of the results of the study. 

The project was completed in May, 1990. It must be pointed out that all these systems are constantly being 
modified and improved by their publishers, and what is reported here was what was obsen^d during the 
1989-90 school year. Anyone considering purchase in the future should carefully check for changes and 
improvements in each of the systems which may make them more appropriate for any given setting. 
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StctiMS: GENERAL ISSUES 



There are some considerations in the use of an Integrated Learning System (ILS) which are controlled more 
by the individual program than by the ILS which is used. Because they are controllable by the program, it is 
essential that these factors be examined as a part of the planning process. 

SUfflng/Trmlnlng 

None of the systems examined is so complicated that it requires staffing with experienced computer person- 
nel. The training and documentation provided are adequate to insure a system's effective use. Of course, 
staff chosen for primary lab responsibility ideally should feel comfortable with computers and enthusiastic 
about the contribution they can make to the program. 

Every system needs to have at least one person with primary responsibility for its use, a person with strong 
organizational skills and the ability to work well with students, not necessarily a strong computer back- 
ground. It is not workable to have a number of instructors equally sharing this task; one person should 
monitor the overall use and maintenance of the system and report and correct any difficulties. Beyond that 
requirement, there are several different staffing designs which can be effe'Ttive. 

Technical st^fft One program design uses a technical-type person to act as a lab manager. A person in this 
role primarily monitors the correct use of the system, answers user questions, and provides instructional 
stafT with student performance records. Instructors still need to be available to provide the bulk of supervi- 
sion of student learning activities. 

Clerical staff: Integrated Learning Systems automatically do most student record-keeping, so that a clerical 
staff member may not be needed. Most of the systems have no provision for handling any needed institu- 
tional reports, however, so time must be alloted for that as well 2S student registration, answering the 
telephone, etc. Programs which choose to purchase CCP and to become a member of the U.S. Basics 
collaboration especially need to consider the use of a clerical aide, since members of that collaboration do 
extensive .nformation gathering and reporting. 

Instructional staff: The instructor is the most important factor determining the success or failure of any 
system. Even the most sophisticated integrated learning system does not have the capacity to show students 
how to work effectively with others, and systems are usually somewhat limited in their ability to teach 
problem-solving or to show the application of skills in life situations. This makes it essential for an ILS to be 
complemented by classroom instruction or individual tutoring. Most of the systems recommend that an ILS 
be used as one part of an overall pirogram in which learning is structured by a qualified instructor. 

Training, Some amount of training is provided with the purchase of every system. In most cases, this 
training is provided on-site to as many staff members as will be involved with the use of the system. The 
length of the training varies, but all publisher representatives indicated that if the initial training were 
inadequate, further training could be provided until the staff felt comfortable. The best training model is to 
have up-front training before the system is implemented with students, then additional follow-up training 
after a few weeks of system use, to work out any "bugs." It would be ideal for staff members to have one or 
two days or even a week following the training to work with the system without any students. 

An important consideration for programs with a high staff turnover rate is the availability and cost of 
training for new staff members, as well as whether on-the-job training is sufficient or the company requires 
staff members to complete the company's training program. 



The roorn selected to house the ILS will need to be large enough for the number of work stations purchased 
as well as any additional instructional materials. (In the case of CCP, for example, materials include video 
tape players and extensive print libraries.) The room will need to have adequate electrical service for all of 
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the work stations and be fairly secure and well ventilated. Chalk boards are not good co-habitants with 
computer systems. One lab manager suggested that a good addition to a system would be keyboard "skins," 
which make keyboards much less susceptible to dust, cookie crumbs, or coffee spills. 

The management station could be housed in the same room or in an adjacent room. Having it in the same 
room makes it easier for the lab manager/instructor to interact with students, while housing it separately 
makes it more difficult for students to enter the managem.ent system and alter records. Though most 
systems have a '^password" to provide some security against system break-ins, if security is a concern it would 
probably be worth considering putting the management station in a secure location. 

Systems which require a modem for communication with the corporation or for management of remote sites 
will need to have a telephone line. 

Schtdnllflg 

There is no attendance pattern or schedule inherent in these systems. It is possible for students to attend at 
any time, for any length of time. The attendance pattern of students and the amount of time spent on the 
computer and on other activities, combined with the ability to provide staffing, would determine the number 
of students who could be served. The systems themselves are completely flexible and could operate 24 hours 
a day. 

The student workstations are usable with other, non-system software so tiiat classes such as word processing 
or computer programming could also be scheduled. (Software which can be used is dependent upon the 
hardware and software configurations of a system — see individual reports.) 

Sections: SPECIFIC ISSUES EXAMINED 

These are areas which need to be considered when looking at any large system. The individual system 
reports which follow include sections for each of these. 

Brief DcMription of Syttem 

An overview of the system, including any unique features and the name of the system publisher. 
Prognm Where Observed 

For this report, each publisher recommended the Washington program which provides the best utiliwtiui! of 
the system, and the individual reports are based on observations of those programs. Information is included 
about student population, program si?e, staffing, and length of time system has been in place. 

Cost Factors 

In this report, cost information is intended for comparison and estimation purposes only. The design of the 
various systems with their individual elements means that this type of comparison should be used only to get 
a vei7 general idea of tiie approximate cost. Publishers provided cost information for a typical eight-station 
lab with a teacher management station and software for an ABEyCED program. CNS provided information 
about tiieir software modules, but not for hardware, since that must be purchased separately. CCP did 
provide comparative information, but the system can vary tremendously — a progi'am would not have to 
have ami student work stations if it utilized only print materials, or could choose to buy a wide range of 
instructional materials, including not only print but also computers, software, VCR's and videotapes, cassette 
recorders and tapes, language masters, etc. 

Another cost consideration v^hich is important for ABE programs, where funding is often problematic, is that 
most publishers have lease options for their systems, making it possible to fund an ILS over the course of 
several years. 
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Most of the systems have more than one possible hardware configuration, and most have enough flexibihty 
to allow the use of previously owned computers for student work stations if they are of the right type. In 
addition to a management station and student work stations, systems usually require a printer and some 
need extra components such as a speech synthesizer or an optical scanner. 

Qttalily ofSyHegn 

Consideration should be given to the durability and quality of the hardware and software components as well 
as information about graphics and sound, sophistication of the instructional material, and quality of the 
system documentation manuals. 

AifiW9tfaMU nbovt Uaraing . 

Questions to consider include whether the learning is student-managed or instructor-managed, whether m- 
struction is computer-based or multi-resource, whether it is competency-based or .vorm-rcferenad, and 
whether instructional material is based on mastery of discrete items or integrates knowledge and transfers 
learning. 

latlrMtloiial Dcalgii 

Systems vary in their ai ;as of instruction, question type, and clarity of learning objectives. The most 
important consideration in this section is whether learning objectives are presented sequentially, from a 
menu; or whether the system is capable of "branching" a student into review materials or more advanced 
materials, and of "spiraling'' a student through objectives, introducing a topic and then returning to it for 
further instruction or review. 

In a sequential presentation, a student would select, for example, a lesson on capitalization. The student 
would work on that unit until mastery was achieved, perhaps being given extra exercises if he or she were 
having difficulty, or perhaps simply repeating the lesson until it was mastered. Once the unit was complete, 
the student would probably not see further questions on capitalization unless there was a "section review" 
type of lesson. 

In a branching, spiraled presentation, however, the student would simply choose to work on a general 
concept such as language arts, and capitalization would be presented at varying points within the curricu- 
lum, initially being introdu ced, then later being taught, and eventually being periodically reviewed. If the 
student had difficulty with capitalization, instructional tasks dealing with capitalization would continue to be 
woven into the presentation until the student consistently performed well, at which time it would be pre- 
sented as a review concept. 

AMtMment/PlMemciit 

Some systems have built-in assessment and placement instruments which automatically measure a student's 
performance level and then move the student into appropriate instruction. Other systems require that the 
student complete a pre-test and that the instructor make assignments based on the pre-test performance. 
Additionally, systems have different ways of assessing student progress within the curriculum. 

Inttractloa^ DtUvtry 

This section is probably the longest in most of the reports. It includes general mformation about what cur- 
riculum materials are available, what types of instructional matenals are used, and who the intended audi- 
ence is. Then it looks at the particular instructional materials, how they provide mstruction, and how 
effective they are. 

Ftf dback 

fhis is an important attribute of Integrated Learning Systems. They need to provide information to the 
student about b' • responses, correcting effectively when necessary and reinforcing correct responses. They 
vary widely in the sophistication of feedback provided. 
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Utabfllty 

This section considers how "user friendly" Jie system is, including the use of on-screen menus and direc- 
tions, "bookmarks" to remember where the student stopped at the last session, the ability to review instruc- 
tions, the level of typing skill needed, and the ease of generating a report. All systems examined were 
reasonably usable. 

Nodlflablllty 

A number of parts of a system may or may not be modifiable. Some systems allow instructors to add instruc- 
tional software or to reference other instructional materials. Most allow instructors to set the level of 
mastery or the rate of presentation, and some allow instructors to determine such things as whether stu- 
dents can pre-test out of a particular lesson or objective. Additionally, systems can be modified for use for 
other non-system purposes such as word processing or programming classes It's important to recognize 
that a system which can be easily and extensively modified will then be limited in the amount of branching 
and spiraling (see Instructional Design, above) which it can do. 

Nanfticsnent of Stndtnt Pcrfomuuicc 

Different systems have different ways of managing student performance. They have different criteria for 
mastery and different ways of reporting student results. Some systems have the option of re-arranging the 
objectives to match a particular course and elimmating unneeded parts of the curriculum or adding extra 
parts. 

Rtconb Nanagcment 

Issues are how record keeping is done (by hand or by computer), what reports are available, how easy it is to 
use the records management system, and how many records can be maintained within the system. 

Sapport/Scrvlcc 

An important consideration is how easy it is to get service when needed and what types of supports, such as 
additional staff training and a toll-free hotline, are available. 

SUff and Sladent Rcipoiue 

When possible, staff and students were interviewed and asked very general questions about stren'^^hs and 
weaknesses of the system. It's important to remember that these responses are subjective and mv r may 
not be accurate. 
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Sectloo 4: INDIVIDtAL SYSIEN REPORTS 



CCC SYSTEM 



Brief Dtioriptloa of Syftcm 

The CCC Ins^tructional System is a product of the Computer Curriculum Corporation in Palo Alto, California. 
It is a ^rlosed" system which uses only the software which has been integrated into that system. (An IBM- 
based system is also available, which makes the system more open, but precludes thie use of remote sites.) 
There are two types of courses within the system — lessons courses and strand courses. In lessons courses, 
students master a set of objectives one by one. In strand courses, students select a curriculum area such as 
reading, writing, or math, and then work on a number of objectives which are interwoven. The strand 
courses are the more recently developed ones and are much more sophisticated in nearly every way than the 
lessons courses. Each work station has a mouse which can be used to manipulate on-screen tools (such as a 
ruler), draw lines, etc. Some unique features of CCC are that it can continuously adapt instruction to an 
individual learner's performance, can provide some parts of the ESL instruction in any one of six foreign 
languages, can predict how long it will take a student to reach a goal, can print out supplemental worksheets 
for a student based on that student's current study, and can be used for electronic mail, sending messages 
from the instructor to one or more students on the computer screen. 

Prognni Whm Obitrvcd 

CCC was observed at the Washington Institute of Applied Technology. It has been in place 
six months, and is used with 40 out-of-school youth and 40 adults daily. It is used as the sole provider of 
basic skills instruction. Students spend about 1 hour daily on the system, then may use other hardware 
(Macs and Apple II's) and non-system software. WIAT screens students upon entry into one of three catego- 
ries according to their performance on the TABE: students with an 8.5+ grade level go directly into a 
vocational class, students with 6.5-8.5 work on the CCC system concurrent with vocational program enroll- 
ment, and students with levels below 6.5 are assigned only to the learning center and the CCC system. The 
Learning Center is staffed by a system manager, a GED instructor, and an instructional aide. 

CotI Factors 

The estimated cost for 8 work stations, all necessary hardware (Atari work stations), a selection of basic skills 
software, and set-up and training is about $53,000. It is possible to purchase fewer or mo/e modules and 
different hardware options, thereby affecting the cost Both the software and hardware components can be 
easily expanded. If a Microhost is used as the file server, it can support up to 128 stations (including remotes 
connected by modem). Also available at extra cost are penpherals such as a speech synthesizer and graphics 
server. 

Hardware 

CCC can utilize a variety of computer types, including IBM, Atari, and (with some courseware limitations) 
Apple He and Ilgs. Some of the courseware requires color monitors and graphics capability, so this must be 
considered in making hardware decisions. The file server control unit has a 128 megabyte hard drive with a 
Unix operating systerr which can control up to 128 work stations, including remote sites, and can hold 5,000 
individual records at one time. The system uses RGB graphics and a Unix-based Microhost speech synthe- 
sizer. Each work station has a mouse, so that students can use pull-down menus and icons. 

Another option which is available is an IBM I-platform control unit. The choice of this option means that 
any DOS software can be listed on the menu, but it also means that the control unit cannot service remote 
sites. 

An interesting option which has recently become available is something called the SOLO (^tand-alone 
Option for Learning Opportunity). This is a single 286 IBM PS/2 which has audio and all of the ABE, GED, 
and ESL cunric um modules installed on a CD-ROM (compact disk). The current cost of the SOLO is $6280. 
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QaalltyofSyittm 

The software falls into two categories: strand courses 3ind lesson courses. The quality level of these two 
courses differs greatly. The strand courses use sophisticated color graphics, a mouse, help icons, and, in 
some cas^. audio. The lesson courses use none of these. The strand courses have attractive, easy to read 
screens with white backgrounds and color Images, The lesson cout^s have black backgrounds with white 
letters which can be difficult to read. 

The graphics, when used, are sometimes slightly juvenile (monkeys and alligators) but not really offensive for 
adult learners. The audio, when used, is adequate rather than outstanding. Generally the content is 
accurate and free of "bugs" and errors, although an instructor commented that toward the upper end of the 
curriculum, there are occasional problems with screens ''freezing" or not accepting student answers. 

The system documentation is well done, well organized, and easily used by a trained user of the system. It is 
unlikely that an update would destroy student records, but WIAT recommends doing a back-up of student 
records and re-installing after the update on the remote chance that the update could have an effect. 

The Microhost system has the reputation of being very stable, and the hardware is of high quality. 
AssQinptloiu aboat LcumlnK 

CCC is a computer-based system, rather than multi-resource. Students who cannot or will not use a com- 
puter would not be able to use the system. It is a competency-based system, in which learners progress by 
mastering competencies rather than achieving a norm-referenced performance level. In the lessons courses, 
these competencies are taught and tested discretely, with no real provision for integration of knowledge or 
transfer of learning. The strands courses, however, use a more integrated approach. As a student moves 
through a strand course, different competencies are woven together, and previously taught competencies are 
reviewed. The use of such on-screen tools as a thermometer or a ruler increase the likelihood that learning 
can be transferred. Because of the integrated nature of the strands courses, a learner simply selects the area 
for study (such as reading or math) but cannot choose a specific lesson. In the lessons courses, a learner sees 
a menu and selects the lesijn. Learners can be given varying degrees of control over the sequencing of 
lessons courses and, if given total control, could complete the lessons in random order. 

IiutnictioiuJ Design 

The "lessons" courses are similar to stand-alone CAI. The presentation is sequential, with movement 
through the curriculum based on mastery objective by objective. Although learner objectives are not sUted 
directly, the purpose is defined and students generally know what is expected of them. Questions are almost 
-exclusively multiple choice. 

The "strand" courses have much more complex branching. The presentation is seouential. but there is 
frequent review and constant adjustment accordmg to student performance. Question types are highly 
vanec. 

The system is able to provide instruction in K-12 and ABE math, reading, and language arts; secondary-level 
math, reading, language arts, and computer science; GED preparation; and ESL with instructions provided in 
Hmong, Spanish, Japanese, Mandarin Chinese, Arabic, or Italian. 

AMCMment/PUcement 

Students can be placed into courses such as math or reading or writing by grade level or by objectives to be 
completed. In either case, diagnosis is done outside of the system (WIAT uses the TABE) and initial place- 
ment into the system is done manually. The instructor assigns courses for each entering student, up to a 
maximum of five courses per student. The instructor has the flexibility either to program a sequence or to 
overricie the sequencing and let the student choose the order. The instructor can also omit certain lessons 
from a course. 

In the strand courses, the system then goes through an 'Initial Placement Motion." This means that student 
performance is closely monitored and the computer gradually "narrows in" on the student's actual level of 
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performance. Lessons courses are usually assigned based on a student's need to complete the objectives 
within those coursu, and so the student simply moves through the objectives sequentially. 

A unique feature of the CCC system is its Individualised Prescription Strategy (^PS). If this optional compo- 
nent is purchased (for about $10,000), the system will monitor student performance, compare it with the 
corporation's yeara of data, and project and provide a continuously updated estimate of the amount of CAI 
time a student will need to reach his or her stated goal. 

The assessment of progress is a continuous process in the strands courses, with the system constantly 
adjusting the level of material to met the student's current need. Another unique feature of the CCC system 
is that mastery is not just percent correct but also amount of time spent on each question. In most sys- 
tems, a student who can answer four of five questions correctly, for example, would be considered to have 
mastered the skill even if he or she took more than an hour to answer those four questions. Mastery in CCC 
means that the student has achieved the necessary percentage correct and has done it within the time limits. 

WIAT has observed that stuaents generally are gaining one month in grade level in reading or math for every 
two hours on the system. A Louisiana study with high risk students age 14 and above showed an average 
gain of 1.5 grade levels after 15 hours of CAI math and 1.2 grade levels after 14 hours of CAI reading. 

lAftractkmal Dtlhtiy 

The CCC System includes 665 Iiours of mathematics instruction, 50 hours of "survival skills," 80 hours of 
GEO Preparation, 467 hours of reading (some with a speech component which require the optional speech 

nthesizer), 445 hours of spelling and writing, 10 hours of keyboard skills, and 310 hours of computer 
program Tiing in courses which CCC considers appropriate for adults, but which would not be a part of a 
bas^'c skills curriculum. Additionally, there is a 35 hour "reading readiness" program which is not listed as 
an adult course but which looks as though it ££tulll be useful with non-readers. About 2/3 of the math and 
reading courses, those at the lower levels, are "strand" type, and upper level math and reading and GED 
Preparation are the "lesson** courses. The curriculum materials are revised yearly, and more strand courses 
are gradually being introduced. 

As the CCC Competencies are similar in many cases to the Washington State Core Competencies Curricu- 
lum, CCC could be useful for Washington ABE programs. It would be difficult to integrate completely, 
however, since specific competencies and objectives <:annot be isolated in the Strands courses, which are a 
m^ority of the curriculum at the lower levels. The materials targeted specifically for the GED are not nearly 
as complete, and typically require the student to read a passage from the CCC GED text, then complete on- 
screen questions. The GED course does reflect the content of the 1988 GED, and there arc on-screen 
practice tests available. A word processor and writing tutorial seem appropriate for GED level students. The 
ESL curriculu''* has a unique feature — it provides spoken instructions in Spanish. Mandarin Chmcsc, 
Arabic, Italian, Hmong, and Japanese. Minimal reading skills are assumed. 

The delivery system is exclusively CAI, with the exception of the auxiliary textbook used in the GED course. 
In the strand courses, it does a wonderful job of taking advantage of the computer's capabilities. The 
materials were developed exclusively for the CCC system and are periodically updated. 

A feature of the CCC system which seems as though it would be particularly useful is its ability to generate 
and print worksheets for individual students or for a group, based on current areas of difficulty. This is done 
through a teacher menu which gives the teacher alternatives such as answers, no answers, odd answers only, 
or answers printed on a separate sheet. 

The bookmark feature which takes the student back into the lesson at the place last completed makes it easy 
for a student to work on the system for any increment of time. Students tan select the curriculum area they 
wish to work on and can alternate within a session between areas such as math and spelling, etc. However, 
because of the nature of the strand courses, moving backwards within a lesson or repeating when desired is 
usually not possible. 
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When a new concept is being introduced, or when a student is having difficulty with a concept, a Kelp icon 
becomes active. Choosing this icon leads to a tutorial on the concept being studied. 

No discrepancies were noted between level of instructions and actuai instructional material, and programs 
seem appropriate for adults (with a minor concern about some graphics — see Quality of System, above.) 

In the strand courses, not only question type but also question topic varies. For example, a student in a math 
course might have one question about adding decimals, then the next question could require application of 
the skill in an aru such as measurement, then back to decimals. Question types include typing in an answer, 
or using a pointer to choose an answer, or drawing a line between matching aa5wen, and so on. In some 
cases, 'tools'* such as rulers are available. A student can move the ruler around the screen to measure 
various on-screen objects. Review questions are also incorporated. This variety of presentation undoubtedly 
adds to student interest levels. 

FttAMk 

A correct answer usually generates a graphic reward — a blue, red, or green ribbon, depending on the length 
of time and number of tries required to reach the answer. An incorrect answer gets a hint, such as He-read 
the first sentence of the paragraph," and a message to try again; a second incorrect answer is followed by the 
correct answer. In language, when an answer was supplied without the necessary capital letter, th^ system 
prompted, ''The Lneeds to be capitalized." In math, any digits which ate supplied by the system are in red, 
while digits supplied by the student are in black. There is usually a '^elp" icon available, which will give 
the student either a hint or the correct answer. These types of feedback are immediate and always have a 
positive tone. 

At the end of a session, students are given a graphic display of their current performance level; usually a 
"thermometer" with the performance level displayed. 

Unbfllty 

Although implementing the system requires little knowledge of computers, it is so complex that training is 
vital. In order to use it effectively, an instructor or lab manager would need not only the company-supplied 
training but also a few days without students in order to become familiar with its operation. 

Orienting a student to the system is less complex. It requires about 1/2 hour to become familiar with the 
function keys and the overall format There is a handbook, which WIAT is considering re-writing at a level 
appropriate for student use. Some of the programs at the beginning levels have orientation to computer use. 

In lessons courses, the instructional system begins with a menu. In the strand courses, the system automati- 
cally enters a student at the beginning of the next objective to be mastered. In order to return to the menu, a 
student laust exit the course. Most of the courses have on-screen directions which can be accessed either 
with a keystroke or by pointing the mouse to the "owl" icon. It's not always possible to repeat screens or to 
review instructions, but in some cases there are on-screen directions to enable the student to do that. Help 
is available in some courses, usually the strand courses, and when it is available, the "help'* icon is present. 

A bookmark feature returns the student to the lesson which he is currently working on. 

In order to have new software be a part of the instructional system, it would have to be re-programmed, a 
process which woul(^ be difficult, if not impossible, for most sites. It is simpler just to disconnect (a simple 
task) the workstation from the system and use it as a stand-alone. WIAT uses non-system Macs and Apple 
He's for students to use when working on non-system software. If a program wishes to use the work stations 
for non-system software for something like a supplemental word processing class, it only requirts a simple 
disconnection at each work station. 

It isn't possible to change the level required for mastery, which includes not only percentage correct, but rate 
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at which a student completes a task. Instead, an instructor would change the student assignment to a higher 
or lower grade level. 

Nmnfti^jMnt ofStadtnl Ptrfomanct 

The strand courses have a highly sophisticated system for managing student p<;rfomiance. Initial assessment 
(see Assessment/Placement) involves an initial placement motion which monitors a student's first 100 
minutes on the system closely in order to determine correct placement. Then continued progress through 
the system is dependent upon student performance, continually being adjusted up and down, depending not 
only OP. number correct, but also on length of time needed for each answer. Because progress is a continu- 
ous movement rather than a series of steps forward, it can be difficult for a student to recognize progress. To 
combat this, students need to receive regular progress reports from the instructor. 

While strands courses have a continuous process of branching a student through the curriculum based on 
performance, lessons courses arc more standard — students complete a pretest, tutorial, and posttest for 
each objective. If mastery is not achieved, the instructor needs to intervene and provide alternative instruc- 
tion. 

Students are assigned only those parts of the curriculum which will meet their needs, up to a maximum of 
five courses at any ore time. Unneeded areas of the curriculum can be eliminated on an individual assign* 
ment basis. 

If a student or a program wishes to have a prediction about the length of time that will be needed to reach a 
stated goal, the optional IPS module (see Assessment/Placement) can provide that information. 

Rccordi Naiui|cnenl 

CCC provides three types of reports which summarize current performance and progress. A course report 
gives detailed information about the student's activity during the most recent session, a grouping report 
highlights areas where the student is experiencing difficulty (which can be used to group students with 
similar difficulties), and a gains report (strand courses only) 

sunrimarizes student gains in level since placement and in the most recent 20 sessions. The gains report can 
also be requested for a group of students to show the mean gain and time spent. 

The management system is rigidly structured and does not allow for alternati^^e formats or provide other 
types of reports, such as scheduling or attendance. These institutional rcquirementi wcv^u > be 
completed in another way. 

Adding, deleting,or printing student records can be done easily from the lejchei Menu, but once an item is 
entered, it's n "essary to begin the routine again in order to change that ^teip. These functions are menu- 
dnven and, once the codes and curriculum are learned, easy to do. 

Anyone who has the necessary codes and passwords couk Alter or delete records. For this reason, passwords 
can easily be changed and CCC recommends that this be uone frequently. 

One central file server control unit can serve up to 128 work stations, including remote sites connected by 
modem. It's also possible for a central office to be connected by modem to schools with CCC systems in 
order to monitor student performance reports. 

Sapporl/Scrvlce 

CCC has a hotline, which has been very helpful to the WIAT Learning Center. There are also local techni- 
cians available when necessary. WIAT initially had some set-up problems, but they have been resolved 
satisfactorily. Additionally, a weekend power outage caused the system to crash, but it took only about half 
an hour of telephone assistance on Monday morning to get back on line 

Staff training is provided with purchase and is available on an on*going basis as a part of the licensing 
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agreement, hut may not be needed. There is no user support gioup or conference schedule. 
Stodtat SmcUom 

Students interviewed had mostly positive reactions to the CCC system. They like the detailed instruction, ti.? 
fact that learning keyboarding skills is a ^nus " and the speed at which progress is made. Two students 
complained that occasionally the computer doesn't seem to recognize a correct answer, or that the same 
lesson is repeated. It wasn't possible to verify whether these were legitimate concerns or a reflection of the 
students' difficulty understanding some of the lessons. 

StmfrinctloM 

Instructors are excited about the system. They appreciate the reports available, especially the grouping 
reports which make it easier to organize small group instruction. One instructor regularly takes advantage 
of the provision for sending "electronic mail'' to a student — when the student begins a session, he/she 
receives a ''message waiting" signal, and finds a message of encouragement from the instructor. 

Only two negative comments were maoe — that the strand reading courses top out at about a 7.5 grade level 
(there are lessons courses available for higher grade levels), and that at the upper end of the curriculum, 
there are some minor system problems. 



CCP SYSTEM 

Brief Description of Syttcm 

CCP, The Comprehensive Competences Program, is distributed by U.S. BASICS, the United States Basic 
Skills Investment Corporation, of Alexandria, Virginia. It varies significantly from other large-scale Inte- 
grated Learning Systems in that it is primarily a learning management system which references a large 
number of materials and a variety of media, including textbook5, workbooks, cassette tapes, and videos in 
addition to software. 

On the CCP system, a student takes a paper/pencil pretest which is scanned, and a diagnostic profile of 
strengths and weaknesses is printed out. The student and teacher then conference to determine priority 
areas of study and the teacher makes assignments using the CCP list of recommended materials. After 
completing these assignments, the student takes a paper/pencil test which is again scanned and progress 
sheets are pi inted out. 

U.S. Basics is a not-for-profit corpor^.tion, funded primarily by the Charles Stewart Mott, UPS, and Ford 
Foundations. Potential users may choose to prepare and submit a formal application to U.S. Basics to be 
approved as a BASICS Center for continuing program support and participation in regional training activi- 
ties, or to purchase the system outright. For collaboration members, program effectiveness is monitored 
through quarterly reviews, iuid annual renewal of the CCP franchise requires that performance standards are 
maintained. 

CCP released a newly designed system, using an IBM management station with a hard drive, in late spring 
1990, just as tile final report of this project was being prepared. It is the errlier version of CCP which was 
examined for this project, and upon ^^hich most of the comments are based. This earlier version is no longer 
available for purchase, and U.S. BASICS feels that tiieir new system has many advantages over the old. The 
basic program design has not changed, however, only those aspects which relate directly to the use of a hard 
drive rather than floppy disks. 

Program Whcrt Obtmed 

CCP was observed at Skagit Valley College, where the system has been in place 2 1/2 years, serving 80 ABE, 
GED, and Developmental Ed. students each quarter. The CCP system provides the overall structure, but 
Skagit Valley supplements it with individual tutoring and non-CCP materials. An instructor does most of the 
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monitoring of student performance and supplemental instruction, and a half time clerical aide does the data 
entry and test scanning. The learning center has six student computers, which are used to provide about 5- 
10% of the instruction. 

CottPictort 

Information about a typical CCP lab purchase was provided. This ''typical" lab consisted of a management 
station with necessary software. 5 student work stations with $9,000 worth of software. 3 VHS recorders/ 
players. 5 audiocassette players, and appropriate instructional audiocassettes and videotapes, a library of print 
materials (2-8 copies of each title), and the implementation/training package. Such a typical lab would cost 
approximately $49,000. Because many of the materials which are referenced by the CCP system are those 
which are very common in adult education, many programs may find that they already own some of what is 
needed. That equipment and those materials which are not already owned may be purchased either from 
U.S. BASICS or from otiier vendors and publishers. In 1989. purchases from U.S. BASICS ranged from 
$10,000 to $100,000 with the average purchase roughly $35,000. The annual user fee is $1,000 for members 
of the collaboration, but for that user fee U.S. BASICS provides not only support but also $1000 :n coupons 
which can be used to cover up to 25% of the price of additionally purchased equipment or materials. 

Hjurdwurt 

The hi^rdware utilized for the management workstation has been either an Apple He with three disk drives or 
an AppleSider hard drive. New systems utilize an IBM with a hard drive. If available, a nK>dem can be used 
tor communication with U.S. BASICS' bulletin board. Some of the newer instructional software in the 
system requires color monitors, so whey would be preferred if a program is acquiring new hardware. Because 
the student work stations are not networked or integrated with the management workstation, the type of 
student computers used is more dependent upon the software chosen than on the CCP management system. 

QnalltyofSytlefli 

The Apple He with floppy disks can be inconvenient and the AppleSider hard drive has been unsatisfactory in 
several early installations. U.S. BASICS has converted to an IBM hard drive, which they hope will prove 
simpler and more reliable than either of the previous options. 

* major criticism of CCP is that some of the material is badly out of date. New modules are being added, but 
the older modules are still available, which may contribute to this impression. There is a regularly-published 
list of new materials, but many of the books which are referenced in original modules have publication dates 
of 1975 and earlier, and much of the software was developed more than ten years ago. 

The system documentation remains wordy and difficult to use. although it has improved somewhat. Some 
centers have had minor problems With their Apple management software. 

Some programs may choose to use already owned hardware and software, which will of course be a signifi- 
cant factor in the quality of the system. The scanner at Skagit Valley, used for scoring student tests, has 
occasionally needed replacement, some of the computer disk drives are rather balky, and some software has 
"crashed " 

Aitamptloiu dioat Lcwnilng 

CCP is a multi-media system with an eclectic instructional approach because of its use of a variety of media 
and materials. It is the only system of the five which would be appropriate for use with learners who are 
unable or unwilling to use a computer. CCP uses a "clinicar* approach, with assessment, diagnosis, prescrip- 
tion, and follow-up testing. It is a competency-based system, in which learners progress by mastering 
competencies, and this progress is cross*checked with GED practice tests and/or TABE tests. Learner 
objectives are not always clearly spelled out in terms of individual objectives, although students know the 
topic and the requirements of the unit and mastery tests. Learners then take responsibility for finding 
instructional materials by reference number, completing the learning activities, and tracking their own time 
and assignments. It would be possible to complete objectives in any order desired, which could defeat the 
"building block"* approach of mastering simple skills before attempting complex ones. Because one compe- 
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tency might require the use of several different instructional materials, integration of knowledge and transfer 
of learning are probably higher th^ in a strictly CAI system, depending upon the particular materials used. 

It is possible to identify and teach specific isolated competencies, should that be desirable. 

Instractloul Dtdga 

Identified areas of instruction are functional literacy and academic subjects tor Developmental Ed., CED, 
and ESL students. Within each component, there are three tiers: Basic, Intermediate, and Advanced. An 
individual student's prescription is based on his performance on a pretest. A test can be taken at any time 
and IS scored by scanner, then the instructor and student must review the results and determine whether the 
student should move to the next unit or whether the current unit needs review. U.S. BASICS recommends 
80% for mastery, but the instructor makes the determination of whether the student has in fact mastered the 
competencies or not 

AmMaMii/PUctmtal 

The TABE and the GED Pre-tests are used for diagnosis and placement Tests are taken with paper and 
pencil, then answers are scanned into the system. The system can do a diagnostic profile for TABE results 
which indicates grade level equivalence in each subject and predicted CED scores. The TABE is not consid- 
ered useful by most programs as an assessment tool for either ESL students or beginning readers, so 
U.S.BASICS does not require the use of the TABE with these populations. 

When a new student begins, s/he must complete approximately one to three hours of testing with the TABE, 
depending upon the battery of tests used. Following that, the instructor and student spend about ten 
minutes determining which areas will be studied and the instructor makes assignments. New students are 
also given an orientation to the system which takes about ten minutes. 

At the end of each unit, a student takes a paper/pencil multiple choice test provided by U.S. BASICS. In 
general, these assessments do a good job of measuring the skills which have been taught Test answers are 
scanned and recorded by the computer. 

Skagit Valley College has documented an increase of about four grade levels for each 100 hours on the 
system in a given curriculum area. A 1989 U.S. BASICS study showed a 1.3 grade level gain in reading or 
math after 33 hours of study. A Skagit Valley staff member speculated that one possible reason for their 
higher rate of increase is that some of their students are reiearning fairiy stiong basic skills and require only 
a quick review before post-testing. 

InttrvcUoiiAl DtUveiy 

The system includes a range of skills and skill levels, from reading readiness through high school/GED and 
life skills competencies. There are 20 subjects in Developmental Ed/GED and three subjects in ESL. Having 
such a broad range, CCP includes many of the competencies from the Washington Core Competencies 
Curriculum. 

Instruction is delivered using a variety of materials including paper/pencil, videos, audio cassettes, comput- 
ers, and language card readers for ESL. Tutoring, lecture, and discussion could also be integrated into the 
system. Instructional material at the lower levels seems designed for children and is largely inappropriate for 
adults. The reading curriculum is not effective for Level 1 ABE students, according to the Skagit Valley staff, 
and there are no Level 1 writing materials. 

The amount of time spent with the computer depends upon the subject and upon student choice; it could 
even be none. Actual instruction is primarily through print; the computer-assisted portion of the curricu- 
Sum is almost exclusively drill and practice. faS, BASICS feels that this is not the case with theirnewerCAIJ 
Some of the reading materials and the newer GED materials do emphasize higher-order thinking skills. The 
materials used are a collection of programs and materials from various publishers. This use of material from 
a variety of sources can be problematic; not all of the materials sequence skills identically, so a student may 
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encounter infonnation which incorrectly assumes prior knowledge. There are worksheets available which 
could be duplicated for homework assignments, although they were not created speciftally for that purpose. 

Because instruction is delivered in a variety of ways, the length of a unit varies tremendously, but generally 
the units are of appropriate length, and time and instruction between tests is appropriate. Students sec their 
progress when noting scores on individual assignments and unit tests. 

PttAadt 

Feedback for individual assignments depends upon the materials being used. With standard print, video, or 
cassette, the student uses answer keys to discover whether his work is correct. In the software programs, the 
level of feedback depends on the particular program being used, but is generally of the "No.^or "That's 
right!" variety, sometimes with hints or answers. 

Feedback after testing can be immediate, depending only on the ability of the staff to put the conipleted 
answer sheet through the scanner. Once answer sheets have been scanned, students can see their % correct 
and which questions were missed, and students and teachers can review missed ai'iswers to plan remediation. 

UMbfllty 

The overall program is complex enough that it would require extensive training or experience to implement 
and manage. However, the management system is menu-driven and user friendly and requires little or no 
actual computer knowledge. Reports can easily be generated and results are automatically printed out Test 
answers are marked on scannable answer sheets which are then scanned and the responses automatically 
recorded and tallied. It is possible to use the system for attendance, but in order to do that it has been 
necessary to dedicate a computer with a time card to that purpose only. [Thai's no longer true with the IBM 
systemj 

The computer-assisted instructional materials sometimes experience delays which can frustrate students, 
and the management system could be faster. The system examined ran on three Apple floppy disks; the IBM 
hard-drive system should be much faster. 

With the floppy disk system, student records are completely separate from the management system, so are 
unaffected in the case of an update. This may change with the advent of a hard drive system. 

NodlfiabUlty 

New student materials could be referenced into lesson assignments, which is not a difficult task, but it could 
be a lengthy one. If a program had a favorite assignment to teach charts and graphs, for example, it would be 
easy to enter that under one competency, but it would be difficult to incorporate a new reading series 
because every objective being taught would have to be analyzed and entered under the appropriate compe- 
tency. This is the reason that the Skagit Valley program often elects to bypass the system when they are 
using materials which are not already referenced. If other objectives need to be added to the cumculum. 
there is a process which makes it possible. The process is not difficult, but is lengthy, as it would be necessary 
to enter all resou rccs and test answers. It would be possi ble to " repackage" certai n objectives as a separate 
course, but would be easier just to assign a student certain units or subjects. (Although mastery tests 
include items for all objectives in a unit, not just selected ones.) 

Ninaiemtnt of Stadtat Ptrfarmuct 

It's possible and usual to assign only some of the objectives to a given student. Unneeded areas of the cur- 
rinjJum are simply not assigned. Movement through the curriculum is based largely on teacher discretion. 
The system reports % correct and the instructor can decide what % to accept, but U.S. BASICS recommends 
at least 80% for CCP reporting. There are two versions of each paper/pencil test. If a student does not 
receive a test score which the teacher feels is adequate, the teacher reviews the errors with th^ student and 
makes further assignments from a list of supplemental materials or assigns a review of the core material. 
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RtcMrit NaaagMMBt 

Student performance on individual assignments is not tracked on the management system. The system 
tracks student performance by scanning test ans>Mer sheets for unit, level , and subject masteiy tests, includ- 
ing number correct and which specific questions were missed. Test scores may easily be printed and com- 
pared with previous reports, but the computer does not make a comparison. Deleting and changing records 
IS simple, although not really needed, because tests may be taken as many times as necessary. Completion 
information on a student is entered manually. 

The system will also report student hours and grade gain by subject area at 100 hour increments and/or upon 
exit, although with the Apple management system the daU must be entered manually unless there is a 
computer dedicated to attendance. 



CCP users collect and report on program data as well, including staff costs, open hours, student/staff ratio, 
and costs per learner hour. Additionally, it can collect and report on student demographic data. However, all 
of the information must be collected and recorded manually. In the past. U.S. BASICS has required by 
contract that this information must be compiled and reported to the corporation on a quarterly basis They 
are now allowing programs to choose whether to join the BASICS collaboration and participate in the data 
collection process or simply purchase the system without such regulation, supervision, and support 

If students have the teacher code, it's possible for them to access the management system. This teacher code 
israsily changeable. 

Sapport/Scrvlct 

U.S. Basics has always required a quarterly report, although they are now making it optional (see Records 
Management). The new quarterly report form is just one page. Production of the quarterly report inwivcs 
gathenng financial information and information about staff hours, making sure that student records are up 
to date, and running an analysis. The most time-consuming part of generating the report is keeping the 
participant records current. 



There is a users' support group and U.S. Basics holds regular regional and national conferences. There is 
also an 800-hotline for technical support. The system at Skagit has required very little technical service. 

Six days of training for lab staff is required by U.S. BASICS before a program can be approved as a Basic Skills 
Investment Center. If an approved center is having difficulty, U.S. BASICS requires that the program hire a 
certified technical assistance agent to provide further formal training. 

Student Rcactloni 

Five students were interviewed. Their likes include the speed of feedback from diagnostic and unit tests, the 
individual nature of the program, the variety of materials used, and the ability to re-take tests as many times 
as needed. Minor dislikes include software which runs too slowly, often shows a blank screen, and has 
inappropriate graphics. 

Staff Rcactlou 

The program administrator is veiy pleased with the CCP system, saying that it lives uplo its promises. He 
points out that the detailed record-keeping, while it can be seen as a burden, can also be a benefit. Skagit has 
been able to use the statistics generated in successful grant applications. The staff training requirement, also 
a potential burden, insures that the center is operated by a qualified person. U.S. BASICS insures that 
programs which are certified as Basic Skills Investment Centers maintain quality programs, so that those 
centers have the credibility provided by nation?: certification. 

Instructors view the system as being very versatile and able to meet a variety of student need.^ including 
some computer familiarization. They are pleased that it allows immediate feedback without tracher involve- 
ment. They have received prompt and helpful technical support and appreciate the user support group. On 
the other hand, they feel that some of the matenals are not as well done as they should b- , and they regu- 
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larly use non-system materials to teach certain skills. The record-keeping is sometimes a burden, although it 
has become more streamlined. They feel that the lower level reading materials are not age appropriate. 



Bfkf Dttcrlptioa of Sytttni 

ClassWorks, published by Computer Networking Specialists (CNS) in Walla Walla, differs from the PLATO, 
WICAT, and CCC ^ems in that it is primarily a management system for an assortment of software from 
various publishers such as Hartley, Milliken, Random House, and Educational Activities. This system 
organizes the various programs and reports student performance and progress. It is an open system; 
software which an be added to the system is limited only by the ability of the corporation to acquire the 
rights to do so and by the hardware needed to support it, but it must be added by the corporation, not by an 
individual program or instructor. ClassWorks uses a Local Area Network,with an Apple He or IIGS as the 
master computer, and was desi* 5d to be similar to AppleWorks. Purchasers of the system are not buying 
unique software, they are buying a software management tool, a variety of courseware organized into 
meaningful assignments, and the capacity to track and document student progress. 

Progmi Whtrt Obtmtd 

ClassWorks was observed at the Thurston County Youth Service Center in Tumwater. This is a juvenile cor- 
rectional facility, which has about fifty students using the system on a daily basis. They have had the system 
in place for five months, after a trial period of about one Tionth. Most students using the system are earning 
credits for a return to the home school district. Students are on the system for a one hour block of time each 
day as one part of the educational activities provided. They currently use only the math and language arts 
portions of the system. 

Coal Ficton 

The cost of the system can vary widely, depending upon the amount and type of hardware purchased and the 
number of curriculum modules purchased. There is room for flexibility in making purchasing decisions. It's 
possible to use already-owned Apple He's or Ilgs's (not lie) or IBM's, which would substantially reduce the 
cost of acquiring hardware. Othervise, it's necessary to purchase the desired number of student stations, a 
teacher station with a hard drive, and curriculum modules. Prices below are for the management system 
software, the hard drive, and the instructional software, not the computers or printer. The least expensive 
ABE software curriculum module is $21,700 and the most expensive is $43,600. If a program decides to 
upgrade to the more expensive module at any time, the cost of the original module is applied toward the cost 
of the upgrade. CED curriculum modules range from $14,100 to $56,500. There is an optional annual 
service and upgrade fee which is about $1000. There are rental and lease-to-purchase programs. 



The system un support up to 40 student stations, using Apple H's (48K), IBM's, or MS-DOS machines. The 
cost of the overall system will depend to a great extent upon the type of student station selected. ClassWorks 
requires one Apple He or Ilgs with 128K and two 5 1/4" disk drives, a Local Area Network(LAN) with a hard 
drive, and a printer. If the basic reading program is selected, a SuperTalker will be required. 

The operating system is DOS 3.3. There is no real remote capability, although it is possible to locate student 
work stations as far as 4000 feet away from the management station. Student workstations do not have 
graphics interaction capability with the use of a mouse. 

QQillty of Syttcm 

There have been problems with networks in the past which CNS feels have been solved. They say that the 
particular hanl drive they are using has been available for a long time and is very durable. 

Most of the software lessons observed were at least five years old. In most cases, the subject matter has not 
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changed, but advances in programming have made newer pr grams much more sophisticated in terms of 
branching, help menus, and feedback after an answer. However, because the software has been used for 
several years, it generally is free of ''bugs" as well as grammar, spellip«f; punctuation, and usage errors. 

The system can run with either color or monochrome moniton. The systems which were examined had 
color monitors. While these are generally more attractive, in several instances the print on screen was very 
difficult to read. 

Some of the graphics observed are only marginally acceptable for use in adult programs. The Hartley 
programs, for example, are exclusively drill and practice, and use luppy faces" with wiggling ears as a reward 
for a correct answer. Most of the programs do not use audio except for ''beeps" which can be turned off. The 
basic reading program does have sound which, while understandable, is not really clear 

AiMflwtkMU ikottt Uaralag 

Glassworks is a computer-based system, rather than multi-resource. Students who cannot or will not use a 
computer would not be able to use the system. It is a competency-based system, in which 'earners progress 
by mastering competencies rather than achieving a norm-referenced performance level. These competen- 
cies are taught and tested discretely, with little provision for integration of knowledge or transfer of learning. 

The system is programmed to present simple concepts before complex ones, but because of the use of 
software from different companies the sequence is not always completely linear and a particular piece of 
software may incorrectly assume prior knowledge or may repeat concepts taught in another piece of soft- 
ware. StudenU have relatively easy access to a menu which allows them to see their records or to change to 
another subject, but they have no control over the order of lessons within a subject. 

Imtractional Dttiga 

Skills are taught sequentially. If a learner fails to master a lesson, a second (and sometimes third or fourth) 
lesson on that same competency is automatically assigned. The reading component provides basic ph-nics 
instruction as well as reading practice.The math programs observed were primarily drill and practice. 
Questions in many of the lessons observed were multiple choice . 

Glassworks lessons can be sequenced to match specific curriculum guidelines, such as the Washington State 
Gore Gompetencies Gurriculum. The breadth of the available curriculum mi'<es it possible to find instruc- 
tional nnaterials for many of the identified ABE and CED competencies. The ABE language component has 
3ome material which would be usable with some of the ESL competencies. 

AsscMmtnt/PUcenicnt 

There is no actual placement test which is a part of the system, so that most programs will use a test such as 
TABE or GASAS to identify student learning objectives. Within GlassWorks, it is possible to select a mode 
which will give a new student an "overview" lesson in each subject selected and will record those objectives 
which were missed, then cycle him or her through the lessons for those objectives. This placement process 
would be inappropriate for beginning readers or low-level ESL students because they would likely be unable 
to complete any objectives correctly and might become discouraged. It would probably be preferable to 
simply assign them all lessons. 

Progress is assessed by tracking mastery of lessons. Within a lesson, progress is sometimes shown graphi- 
cally, for example, with checks in boxes or bar graphs. Progress records are readily available to teacher and 
student. 

CNS indicates that ClassWorks produces 20-30% improvement over other instructional methods (not other 
CM programs). A JTPA program in Kitsap Gounty showed an increase of 2 grade levels in reading and math 
after 8 weeks. Programs observed had done no studies comparing or measuring student performance. 
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Instnctlottftl DfUvtry 

CNS has available more than 1500 software programs from over 30 publishers. They are also willing to 
discuss customizing their pre-packaged modules to suit the needs of a particular program, but sometimes 
that possibility depends upon their ability to obtain rights to the desired programs. 

CNS has targeted ABE and GED students, and the system would be usable with a range of these students, 
from low-level ABE to GED and, to some extent. Developmental Ed. Additionally, there are life skills lessons, 
vocational education sequences, keyboarding, and word processing programs available. It has limited 
applicability to ESL students. There are some concerns with the appropriateness of the software for use with 
adults (see Quality of System.) 

It's difficult to address the question of whether the system takes advantage of the computer's capabilities, 
since that strongly depends upon which of the many software programs is being examined. However, there is 
a preponderance of drill and practice programs with multiple choice questions and limited branching or 
review. 

The units of instruction, called lessons, are well within the limits of a student's attention span, ranging from 
a minute or two to about 20 minutes in length . A few of the lessons give the student an estimate of the 
length of time it will take him to complete the lesson. If a student should exit the system during a lesson, 
the results from that one lesson are not recorded and the student will begin that lesson again at the begin- 
ning of the next session. Because of the variety of publishers, there is a tremendous variety in presentation 
and question type. This is an advantage in that a student who does not And one lesson helpful can try other 
lessons with other approaches. It's a disadvantage in that there isn't a standard format, so that a student 
needs to adjust to each new type of presentation and question. Most of the lessons seem to have sufficient 
exposure and opportunity for practice. However, most of the programs have a pre-detemriined sequence so 
that a student who is doing well and a student who is doing poorly will be given equal numbers of problems. 

Few of the lessons are sophisticated enough to prevent guessing or random answerin,,, but if it leads to a 
poor score, that will be reflected on the student report. In the case of oiv* language program, the correct 
answer for every question in a sequence of ten questions was "b." The number of trials in most lessons is two 
or three and in some of the lessons, hints are given after incorrect responses. 

As mentioned above, there is no provision within a lesson for automatic branching to adjust difficulty levels 
or sequence according to student performance, and there are no review questions. The system attempts to 
provide this type of branching and review by controlling the student's progress sequence. Usually, if a 
student has an extremely low score on a particular lesson, he is directed to "See the teacher" or to repeat the 
lesson. If the score is low but above a failing level, the student may be given a second lesson on the same 
topic. If the student achieves a mastery score, he is given a lesson in a new topic. 

Higher-order thinking skills do not seem to be addressed by most of the software programs in the system, 
although the programs observed were primarily at a low level. 

Fetdbftdr 

The fet^back given for correct and incorrect answers depends on the particular lesson. It frequently seems 
limited to 'That's right!'* or "No" type answers. Some of the language programs did provide "hints" after an 
incorrect response and a request that the student try again. 

Feedback in all observed cases was prompt and did not threaten learners or reward incorrect responses. For 
some lessons, it was also corrective and informative. Because of the variety of software programs used, there 
is a variety of types of feedback, so that it does not become bonng. 

Instructors and students need little computer knowledge to use the system. Of course, before the system can 
be implemented, training is needed for staff who will be using it, and that training is provided with the 
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purchase. Subsequent training necessitated by staff turnover can probably be provided on the job by other 
staff who are familiar with the system. Both instructional and management systems begin with a menu, and 
all teacher tasks can be selected from various menus. 

A student can easily return to the menu at any time and choose either to see his score or move to another 
subject, but if he exits a lesson before completing it, the records for that lesson are lost and he must begin it 
again at the next session. It is generally not possible for a student to review instructions and previous frames 
of a lesson without exiting the lesson and beginning again. 

Very few of the programs have help screens available. The pace of the lessons observed was tedious: the 
computer responded slowly to the keyboard, there was a lengthy delay at the start of lessons, and there was a 
wait of several seconds between saeens. 

Instructors can generate reports from a menu. The instructor screens are prompt and printing can be done 
without delay. 

ModlfteUUty 

There is a good deal of flexibility within the scope of the curriculum which CNS has available. Software from 
their library^ can be added (and in some cases, may necessitate an adjustment in cost.) An instructor 
cannot add software to the management system, however; that must be done by the CNS programmers. 
Networkable software can be loaded onto the hard drive which can then be accessed through the network but 
not managed by ClassWorks. Also, a two-stroke command will fake a student off the network to use the disk 
drive with other software. 

Student perfonmiance parameters such as rate or mastery level generally have a default setting, but that can 
be changed by an instructor through the teacher menu. 

MuagtfliMt of Shidtnt Perfonnanct 

Up to 2000 students can each be enrolled in up to 9 different classes, with space for 50 lesson results and 10 
individual data summaries per class. Up to 400 skill objectives may be chosen for classes or individual 
students or may be automatically programmed on the basis of pretest performance. 

Unneeded areas of the curriculum may be eliminated and others added, within the ClassWorks "library" of 
programs. It is possible to select certain objectives and "repackage" them as a separate course. The instruc- 
tor can set mastery levels from the teacher menu or accept the default level for mastery. In both cases, 
mastery is equated with a percentage, not with time. The system does, however, record the amount of time a 
student takes to complete a lesson. 

Objectives are taught with the use of several different software programs from different publishers. Students 
see a menu which lists the objectives (not the specific software programs) they must master. Movement 
through that menu is based on mastery. If the "advance flag" is set, and a student masters the first lesson for 
an objective, the system advances to the next assigned objective. If the student fails to master the first 
lesson, he will proceed through the lessons in the objective. If the "review flag" is set, a student who has 
difficulty will be assigned a reviev; lesson. 

RtcoHU Nanagtflicat 

Reports available are : assignment results (% correct and time on a lesson), class roster, class status, class 
summaries, mastered objectives, and objective range (range of mastered assignment objectives.) A student 
mastery matrix can be viewed by students to increase motivation. There is no formal means for tracking 
demographic data, but there are two empty fields and student ID numbers which could be used for codes. 

Adding and deleting students, printing records, and changing or deleting records are simple, menu-driven 
processes. There is no record-keeping which is mandated by the system. The only manual entry is done 
when making student assignments. 
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Most student records can be accessed only frcTVi the teacher station, using ^ code which does not show on the 
screen. If the code is broken, it can be easily changed. 

SoRport/Smkt 

There is a 800# hotline lu provide answers to user questions. Also, CNS technicians can take control of a 
system via modem to make many system corrections. Users have reported no problems with their systems 
other than a few minor "start-up"* glitches. Staff development is included with the system purchase. 

SUfTSeactioM 

The staff person interviewed was enthusiastic about the use of computer-assisted instruction and felt that 
having a management system such as ClassWorks enhances her ability to monitor student progress. She 
likes the variety of software chosen by CNS, saying that it helps to hold student attention. Her population, 
ju*i^nile offenders, is not a group which would be expected to be enthusiastic about school, but she finds that 
they are eager to work on the computers and that all of them have experienced at least some success. 



Brkf DtacrlptiM of Syflcm 

The PLATO system is a product of Control Data which has recently been acquired by The Roach Organization 
in Rolling Meadows, Illinois. It is an open architecture system, which means that any software from the 
PLATO library may be completely integrated into the management system and it is possible to add non- 
PLATO software to the management system, although the record keeping for this type of software is not as 
detailed. Individual lessons may be selected and grouped in the management system as a particular course. 
For example, appropnate lessons could be seleaed and listed as Tre-GED Reading.'* PLATO is designed to 
provide only computer-assisted inslrjction, although notebooks of supplemental worksheets are provided for 
duplication if desired. 

PrognuB Whm Obicrvcd 

The PLATO System was observed at Maywood School in the Highline School District in South King County. 
This school primarily serves alternative high school students, about 150-175 each month. The PLATO 
system, which has been in place for approximately 1 1/2 years, can be either supplemental to or a substitute 
for a regular high school program. Students in the program may be in-school suspension students, GED 
students, students who are waiting for the beginnmg of a quarter to re-enter a standard high school, seniors 
who are deficient in credits for graduation, or students who are completmg a SAT-Preparation course. The 
lab is also used with a word processing class and is projected to provide parent education at a remote site. 

The lab consists of thirty student stations and a manager station with a file server. The manager ts a techni- 
cian rather than an instructor. In the same building are alternative classr oms, staffed by certified teachers, 
and there is a building administrator. The lab manager does most of the interaction with both the system 
and students working in the lab, although the teachers often come into the lab to supervise their students 
who are using the lab. Students not in a structured alternative program attend the lab on a drop-in basis. 

CotI FacKnrt 

There are several purchase options for PLATO. The least expensive option at the time of this report was an 
eight-station lab using already-owned computers for student workstations. The cost for this type of lab, with 
ABE, GED, and Coping Skills software, five days of staff training, and a 386 Control Center and printer would 
be about $42,000 the first year and an additional $4800 licensing fee to be paid in each of the next four years. 
Tne same system, with a paid-up licensing fee, would be about $56,000. Having The Roach Organization 
provide student workstations would increase the cost to $53,000 ($67,000 with paid-up licensing.) 

A larger system, with 20 new student workstations and the same 386 Control Center, dot matrix printer, and 
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ABE, GED, and Coping Skills Software, costs $89,600 the first year with a $12,000 licensing fee for each of 
the subsequent four years or $125,600 with paid-up licensing 

The system has a 386-based "control center^ which, according to the publisher, can handle up to 100 work- 
stations, but is much nwre efficient when handling no more than 50. A second hard disk with 1000 mega- 
bytes may be added for increased capacity. It also has available an administrator workstation, a 286-based 
dedicated processor. 

Student stations an be Control Data workstations or other PC-compatibles, including IBM PS/2's. Either 
monochrome or color nnonitors are available. There is the capacity for connecting remote workstations over 
communication lines. 

The operating system is MS-DOS, so that the syste^n can run any DOS software in addition to PLATO 
courseware. The system does not use a mouse or pull-down menus or icons. 

Qvalily ofSysttsi 

Many of the materials seem somewhat dated, but some lessons have been recentb^ updated and are definitely 
better than those which have not. Updates at additional cost are available approximately twice a year. Basic 
skills math, reading, language arts, social studies, and writing have been updated in the last two years. Some 
courses are more extensive than others, and expanding those courses which need it is an ongoing effort 

Both nwnochrome and color monitors were examined. The color monitors are definitely worth the extra 
investment, making the graphics more interesting and some of the instruction easier to understand. The 
graphics are clear, but are less sophisticated than those of some other systems. At the beginning of each 
lesson, the same time-consuming graphic appears. The system docs not take ad'*fantage of the high level 
which is possible witii VGA graphics. There is no audio other than an occasional "beep," but it can be turned 
off if desired. 

The system is generally bug-free, but as with any large system, there are isolated problems. In one unit for 
example, the system won't reconl mastery and the lab manager must enter it in another, it says "Here is the 
correct answer^ but then doesn't give an answer. These small problems will undoubtedly be addressed in 
future updates. 

The documentation which comes with the system is well done and fairly easy for the user to understand. Of 
course, the documentation cannot stand entirely alone; some training on the system is needed. 

Instructional and management systems are reasonably crash-proof. The Highline District experienced only 
one software problem in a year and a half, and that problem «vas intentionally caused by a student with a high 
level of computer knowledge. The corporation assisted the district to increase the level of security within the 
system so that the problem has not recurred. Random system failures have been non-existent. 

AirampUoM abort LMnfaig 

PLATO is a computer-based system, rather than multi-resource. Students who cannot or will not use a 
computer would not be able to use the system. It is a competency-based system, in which learners progress 
by mastering competencies ratiier than achieving i norm-referenced performance level. The competencies 
are taught discretely, rather than in an integrated fashion. A student selects a unit, such as "Using Evidence 
to Support Main Ideas," and completes a tutorial lesson followed by a drill and then by a test. If the student 
perfoinis pooriy on the test he is cycled back into the tutorial and a different version of the drill. Review 
lessons for each unit and applications lessons are an attempt to increase integration and transfer of learning 
The student menu lists the skills in the preferred order, but a student may choose to complete them in any 
order, so that s/he could choose to complete paragraph writing before basic sentence structure. This can be 
circumvented by programming a particular sequence or by setting prerequisite skills. 
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laitnictioul Dtiliii 

The PLATO system uses a sequential approach to learning. StudenU see a menu of assigned lessons and may 
be able to select any one or may be programmed to complete the lessons in the order listed. In some 
lessons, objectives are stated clearly, but in others, only the lesson title gives an indication of the content and 
expectations. Students are in all cases given clear messages about the number of correct answers needed for 
mastery. There are several teacher options for directing a learner's movement through the curriculum. 
Learners can be programmed to complete all assigned objectives, regardless of performance or pretest 
resulU, or can be programmed to pre-test out of lessons they have already mastered and to review lessons 
when performance is low. 

Many of the units are multiple choice, drill and practice. Some of the newer parts of the curriculum, how- 
ever, introduce more variety in tutoring approach and question type, asking the student to estimate an 
answer or fill in a blank. A few of the questions do require higher-order thinking skills and several units have 
"application" lessons which use a life-skills approach. 

The Roach Organization indicates that the instructional scope of the system is extensive, intended for a wide 
range of students (from elementary through college, including life coping skills and parent education). In 
some areas, such as job search and life coping, the curriculum lacked breadth, with only a few lessons and 
little significant content. 

Curriculum materials for ESL are being released this spring but were unavailable for preview. According to 
The Roach Organization, this potuon of the curriculum has digitized sound, begins at a vcr>' low k^ei, ib 
life-skill based, and was developed with the concurrence of a national advisory board. 

Aiift witat/PUctmfnt 

On-line placement instruments are available for ABE. CED, and CASAS, although they are not used by the 
Highline School District and were not available for viewing. 

When new students begin at Maywood, they are shown a few basic keybc. rd strokes (such as Enter and Back), 
shown how to enter their names and passwords, and are then free to beg'.n working on their assignments. 
According to the lab manager, the process takes imder ten minutes and none of the students in that particu- 
lar lab ever has any difficulty. (It's probably worth noting that their students are almost entirely adolescents, 
who in general have more exposure to computer use than older adults.] The PLATO System does not have an 
introductory lesson. 

Each PLATO unit has a pretext, and it is possible to allow students to exit from a lesson when they pass the 
pretest. Progress is determined by the student's ability to complete a post-test with the required percentage 
correct. Results are immediate; once a student misses more than the allowed number of questions, the test 
terminates and the student is cycled back into the lesson. These tests follow the same format as the lesson 
questions, so can be said to measure the skills which have been taught. According to the publisher, students 
advance one grade level in reading for every 30 hours of instruction and one grade level in math for every 
nine hours of instruction. The Highline School District has not studied student results in this way. A 
California study with CAIN (welfare reform) clients showed 3-month increases of Reading, +2.89; Math. 
+3.00; Language, +2.01. Students in that study were 84% male: 52% Hispanic, and 15% African American; 
and generally above a 4th grade reading level. 

Inftractional DtUveiy 

The PLATO system has 1500 hours of instructional materials, including basic skills beginning at a 3rd grade 
reading level, high school courses, instructio.i for CASAS competencies, job search skills, life coping skills, 
parenting skills, and CED materials which match the 1988 CED (matched in content, but not in question 
format). Higher order thinking skills are taught in a few lessons, but are not really emphasized. An ESL 
component has recently been added. Because of this breadth of curriculum and the ability to create custom- 
ized courses, the competencie5 could be ? nged to match the Washington State Core Competencies Cur- 
riculum Project. 



■ Integnted Learning Systems 
Page 24 

Instruction is provided on the computer only, although there are also notebooks of optional worksheets 
which could be 4uplicat«i. The system does not generate and print additional exercises or refer students to 
other media for homework. Units of instruction are approximately 15-20 minutes, so that even students with 
a short attention span should be able to complete at least one lesson at each session. When a student exits, a 
''bookmark* will return that student to the start of the next lesson at the beginning of the next session. 

The instructional materials were developed exclusively for PLATO. Other software can be inserted into the 
system and will be listed on the student menu. The level of the lesson instructions and text seems to be com- 
patible so that a student who needs to study the content probably will be capable of understanding the 
introduction and instructions. However, the content of the le5<^ons in some cases is only marginally appro- 
priate for adults. 

At the beginning of each session, the student sees a menu listing all of his/her assignments, with an indica- 
tion of those assignments which have already been mastered. The student can select any of the lessons from 
that menu to complete, unless the instructor has 'locked" the student into a sequential approach. 

Questions are primarily multiple choice, with a few fill-in. Students in some oases niUSt type in the correct 
answer before they can progress. As an example, when given choice A or B, if the wrong answer B is selected, 
the computer will respond "No," and will flip back to the choice. This will continue indefinitely until the 
student finally selects "A." In other cases, the computer will provide the correct answer after two or three 
wrong guesses. There is no provision for preventing guessing or random answer:; However, the student 
report does show the number of guesses so that an instructor has access to that information. Also, on a test 
a student who misses more than the allowable number of answers is immediately cycled from the test back to 
the begmningof the lesson. 

After initial instruction, there is ample opportunity to practice a skill. Even though exercises have a limited 
number of problems, it is possible to repeat the exercise and see new problems. The practice is very repeti- 
tive, however, with little or no variation in question or answer type. 

Some of the courses take better advantage of a computer's capabi!<ties for branching than do others. Overall, 
the system does not do a good job of this. There is no adjustment of difficulty levels or sequence according to 
student performance, and in most cases there is no provision for incorporating review questions or repeating 
questions which were missed earlier in the lesson. However, the "Problem Solving" lesson at the end of 
some units does provide a cumulative review of that unit. 

A student can see that he is making progress because those .skills which have been mastered are starred on 
the menu at the beginning of each session. 

There are no estimates given for the time to complete portions of this curriculum. The Highline School 
District times typical students whenever adding a new component to the curriculum m order to have an 
estimate of the number of hours needed to complete «...dt component. 

Fttdb&di 

Feedback is generally positive and timely, and feedback neither threatens learners nor rewards incorrect 
answers. However, the feedback is generally not informative. An incorrect answer in a reading lesson 
brought the admonition "Be careful with those details!"" A correct answer usually generates a positive 
comment such as "Yes," "Super," "Itight," etc. and then the system moves to the next question. An incorrect 
answer generates a "No" with a general type comment and then the student either recycles indefinitely 
through the questions and answer or is given 2 tries, then the correct answer. Some of the newer parts of 
the curriculum, however, give the student a "hint" before asking him/her to try again. In a few of the math 
sections, there are "graphic" hints which give the student an illustration to help in visualizing the problem. 
There is no automatic provision for extra practice in areas of difficulty, but the student can ctioose to repeat a 
lesson even if mastery is achieved, and the student will automatically be recycled if mastery is not achieved. 
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The system could be implemented and used by staff with ve?y little computer knowledge and expertise. The 
system is usable with a variety of printers. Both instructional and nunagement systems begin with a menu, 
and on-screen directions are easy to follow. Responses to Questions in most cases require one keystroke, or 
typing one word at the most Returning to the menu or reviewing previous screens is possible at anv time 
with a standard key stroke. Because most of the tasks are very simple, few on-screen directions are present 
or needed. 

There is a "bookmark" feature so that if a student exits a lesson before completing it, he or she will return to 
that same place at the next session. There is a help key which is available in most lessons, but pressing it 
typically returns the student to the tutorial, repeating a general explanation of the content, rather than 
giving specifi c assistance. In a few cases, the student has the choice of tait of help and un select either 
assistance with the content or with keyboard strokes or other types of program directions. 

The management system runs with minimal delays, but the instructional system sometimes asks the student 
to wait for periods which seem excessive. 

Major updates to the system make it necessary to clear the system of all student records; in the case of the 
Highline School District, this made it necessary to wait for summer to update the system in order not to lose 
records for the students who were currently active. However, the publisher made some suggestions about 
alternative ways to maintain records while updating the system. 

NoiUfbbUlty 

New software from the extensive PLATO library is easy to integrate into the system. However, the PLATO 
courseware itself cannot be modified other than to set mastery levels or sequencing. It is possible to add 
other software to the management system, but it is outside a PLATO lesson. If this were done, a student 
menu would have a list of assignments that could include PLATO lessons as well as other pieces of software, 
but they would be listed separately and the only record-keeping for non-PLATO lessons would be lengths and 
dates of sessions and an indicatio i of completion. 

The lab manager or instructor may determine the mastery level for a particular PLATO course when it is 
added to the system. It is also possible, though slightly more complicated, to set mastery le\'els for individual 
students. There are default mastery settings if no modification is made. It is possible to limit the amount of 
time which a student has to answer a question and to specify whether a student must complete all of the 
assigned material or can skip portions of it by passing a pretest. 

The lab manager has essentially three choices for moving students through the system: they can be "locked 
in'' to completing all assigned objectives in the order assigned, students can be given complete control over 
order of completion, or pre-requisitcs can be assigned, so that a student must either pass a pre-test or master 
a previous objective before moving on. 

Nanagcment of Stadcnt Perfomi&iice 

Relevant courses from the PLATO library are put mto the system and new courses may be purchased from 
PLATO at any time. The "pre-packaged" courses appropriate for ABE include Reading (139 hours, grades 3- 
8), Math (95 hours), and Language (112 hours). Individual elements, or lessons, from those PLATO courses, 
could also be re-combined to provide customized courses. For example, certam math, reading, and language 
objectives could be selected and "packaged"* as ABE Level 2. 

When a student enters the system, he is assigned relevant areas of the curriculum, depending upon a 
placement test, upon the need to complete a particular course, a. upon a student interview. The student will 
Chen move through the assigned material in any order (unless a sequence is "locked in*'), seeing tutorials, 
completing practice exercises, and taking tests. Once a test is completed satisfactorily, that particular lesson 
or subskill is marked as Mastered. 
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Mastery is determined by percentage? correct and is usually set for the course or objective rather than the in- 
dividual student (it is possible, with jome manipulation, to set mastery levels for individual students). If a 
student does not pass a test, s/he is cycled back through the same tutorial and/or drill and is then given 
another version of the test There is no 'branching'* which would take a student to an expanded explanation 
of an area of weakness. 

Because each skill is discrete, there is no provision within a lesson for review and reinforcement A student 
continues to %wrk on a particular objective until it has been mastered. Each unit does include a "review" 
lesson for ail the skills taught in that unit Once unit mastery has been certified, those skills are not taught 
or tested again, other than being used as a steo to the completion of more advanced tasks. 

Rtcorii NaMgMMat 

The PLATO system can generate either individual progress reports or class profiles. It is possible to obtain a 
class roster calling up a list of student at the control center and then printing the screen. The system will 
print demographic and statistical information if it is entered, although there is no report format for this. 

The system tracks student time at the computer (although not time on task) as well as student prr>gress. It is 
very simple to add or delete students, to change records, and to print records as often as wished. Most of 
those functions are menu-drven. The Roach Corporation and PLATO require no record-keeping by hand, 
though programs will probably find it necessary to keep additional records, such as attendance, in some 
fashion. 

Each user — students, instructors, and manager — has a password which keeps that file secure from others. 
It is also possible to increase the level of security within the system, but that makes some of the records 
management slightiy more difficult The Highline District did have one instance of a student breaking a code 
and altering some files, so they requested and received assistance in preventing other occurrences. 

Sopport/Strvlct 

There has been little service needed on the Highline system in the year and a half it has been in place, but 
when service was needed, it was provided in a timely fashion. The Service Center, located in San Jose, 
California, has been extremely responcive to the Highline Center, usually arriving the same day or next day at 
the latest when on-site service is reouired. There is also a hotline for help, and the system itself identifies 
problems by number so that when the technician calls for assistance, the problem has already been diag- 
nosed and the service center can usually provide assistance over the telephone. 

There is no formal user support group; the Highline District feels that the hotline provides all the support 
necessary. The PLATO organization does not require any type of reporting or paperwork. 

After the initial three days training on a new system, two more follow-up days of staff trainmg are available 
and it is suggested that they be scheduled after the system has been in place at least a month. 

Sittdkiit rtMtioiu 

The students who were interviewed had not used computers before, so had no basis for comparison. Gener- 
ally they like using the computers because it's possible to work individually, because the system is easy to 
use, and because feedback is immediate. They feel frustrated only very occasionally, altiiough the students 
who had been 'locked in" to a sequence because of Highline's program design expressed frustration that they 
were forced to complete ail portions of the lesson even when they got a perfect score on the pretest. Stu- 
dents feel tiiat this introduction to computer use will be beneficial in future activities, but think that the 
graphics are rather childish and time consuming (they were working with an older, monochrome system) 
and would like to be -^ble to take work home. 

Staff iMctiou 

The lab manager would like to be able to view student screens from the control center and would like the 
system to ^og off after a period oi mactivity in order to prevent students* sitting and staring. 
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The building administrator, who was instrumental in selecting the system, chose it because of its flexibility, 
the scope of the curriculum, and the open architecture which makes it possible to add curriculum and 
hardware components, such as audio or color monitors. He would like to see even more added to the 
curriculum, especially Social Studies and Science, where enhanced graphics could substitute for experimen- 
tal laboratory courses. He is looking forward to increased remote capabilities, which are in the process of 
development 



WICAT SYSTEM 

Brief 9tacriftio« of 9yitea 

WICAT is published by WICAT Systems of Crem, Utah. It might be considered a ''semi-open'' system, since 
any WICAT software can be completely integrated into the management system but non-WICAT software can 
only be listed on the menu. The latest version, which runs on an IBM system, can use any IBM software, and 
non-system software can be listed on the menu, although student perfonfnance will not be monitored or 
recorded. WICAT has assessment instruments which are taken on the computer, including a basic skills test, 
official GEO Practice tests, and a learning styles inventory. These assessments are used in making learning 
prescriptions for individual students. 

Progtaai Whm Obaem< 

The system was observed at Clover Park VTI, in Tacoma , where it has been in place nearly one year. Sixteen 
student work stations provide the bulk of ABE/GED instruction and are available for supplemental work by 
vocational or pre-vocational students. The lab serves abo**t 700 students per year. Staffing was in a transi- 
tional phase at the time of our visit, but generally the lab is staffed with one instructor. Most students seem 
to spend 1-2 hours daily using the system. 

Coit Factors 

The cost for an eight-station lab, including all needed hardware, software, I year warranty, 10 days of training 
(five days initially, then five more days as follow-up during the year), and support and service is $60-65,000, 
depending upon the particular hardware and software selected. In su'.sequent years, the licensing fee is 10- 
13% of the purchase price. This licensing fee covers all needed service, an 800# lK>tlipe,'' system updates, 
and up to five days of staff development. The per-station cost for a lab declines as the number of stations 
increases. For example, 16 stations would be $83-91 .000; 24 stations would be $106-1 14,000; and 32 
stations would be about $130,000. These numbers arc just for comparison — it is not necessary to buy work 
stations in sets of eight. 

Hardwire 

A program purchasing a WICAT system could choose to integrate hardware which is already owned in order 
to save a portion of the purchase price. It is also possible to choose cither monochrome or color monitors, 
depending upon the available resources. Work stations may use IBM, Apple, or Tandy computers. The 
central file server can be expanded to manage up to 64 work stations. It is technically possible to serve 
remote sites with a nfK>dem, but WICAT does not recommend it. Their lessons are so complex that transport- 
ing them by modem would cause a significant delay, which violates their principle of having a maximu m 2- 
second response time to every key stroke. 

The base system varies with the hardware selected, but the newest and most popular configuration runs on 
an IBM I-dass network (IBM model 60). Graphics are either MCGA or VGA, and the system uses digitized 
audio. It will support any of a wide variety of printers. 

The ^em does not use a mouse for graphic interaction with icons and pull-down menus. 
QvalHyof^yitem 

Some modules are more current than others, and these provide more sophisticated graphics, feedback, and 
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branching. Updated materia] is provided as it becomes available as a part of the annual licensing fee. Either 
monochrome or color monitors can be provided, and the color monitors are much superior for taking 
advantage of the graphics available. The publisher claims that the system has "exceptional animated graphics 
and the most sophisticated color in the industry." It's difficult to say whether that claim is justified but the 
graphics are well done. 

The audio can be turned on or off for individuals or for the entire class. Initially, it seemed acceptable, but in 
situations where slight distinctions were important, it was not clear enough. For example, in Reading I, it 
was impossible to distinguish between 'lyacir and 'l)at;' and ^Yat" sounded lilce "sat" (which led to selection 
of an incorrect answer and could create student frustration). In Spelling, students are asked to listen to a 
word and then spell it, but it's necessary to rely heavily on the sentence content to discern what the actual 
word to be spelled is. 

The size of print and screen clarity seems fine, with small exceptions. In the Algebra program, it was very 
difficult to see which operational sign (+ • x +) was intended. 

System documentation is extensive and easy to use for someone who has completed training; it could be 
rather complex for someone without that background. Clover Park has had no difficulty with system 
"crashes," but as a matter of routine the system is backed-up on tape on a daily, weekly, and monthly basis. 

AsMmptlou aboal Ltaralag 

WICAT is a computerJ}ased system, rather than multi-resource. Students who cannot or will not use a 
computer would not be able to use the system. It is a competency-based system, in which learners progress 
by mastering competencies rather than achieving a norm-referenced performance level. These competen- 
cies are taught and tested discretely, but there arc reaMife applications and use of basic skills as tools of com- 
munication, which may encourage the transfer of learning. 

InstnictloiuJ D^tlgii 

WICAT uses a sequential-type presentation, with students mastering one learning objective and then mo\ing 
on to another. The ^stem itself does not require students to complete lessons in a simple before complex 
order, but it's possible for an instructor to assign lessons in a particular sequence, so that a student n^.ust 
complete the beginning lessons before attempting the more advanced ones. There is also extensive branch- 
ing within an objective, providing extra instruction or practice for students who need it. 

Areas of instruction includ; basic skills at levels K-12 and adult, and a typing module which serves to intro- 
duce students to the keyboird. Presentation of instruction is generally appropriate for adults. The lab 
manager at Clover Park VTI feels that materials seem to be higher than stated levels, about +1 grade level. 
The statement of learner obj«!Ctivcs is done through lesson titles rather than being explicitly expressed in the 
lesson. The spelling module is sight-based, and the basic reading cuniculum is a combination of some 
phonics and some memorization. Problem solving is emphasized, especially in the modules which have been 
recently updated. 

AMCMMnt/Placcment 

WICAT has the WICAT Test of Basic Skills, WICAT Skills Assessment Tests, ana WICAT CED tests available 
on-line. A customized test creation option makes it possible for a teacher to make, store, administer, and 
score tests on-line, selecting items from an item bank. The CED tests are short and long forms of the official 
CED practice test. Both the test and diagnosis are on-line, and the system will prescribe lessons for each 
missed objective. Also available are Ability Profile TesU which can be used to determine learning styles and 
a "dynamic placement" option which allows a student to omit an objective or move ahead or back, based on 
performance. These tests are not required for placement into the system; in fact. Clover Pa'-k VTI uses the 
TABE. If an on-line test is used, the system will suggest lessons, but the teacher must enter those which s/he 
wishes to assign. None of the tests gives information about the time and effort necessary to reach a goal 
based on assessment. 
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Placing a student into the system can be done fairly quickly once decisions have been made about what the 
assigned lessons will be. There is no orientation lesson for the student other than a ten minute keyboarding 
lesson which does a good job of introducing the important keys. The assessments within lessons as well as 
the unit tests seem to do a good job of measuring the skills which have been taught According to the 
niiblisher, programs have experienced significant gains in performance levels and decreases m dropout rates, 
in one study, of students who spent at least 60 houn in reading and math remediation, 100% passed the 
corresponding portions of the CED on the first try. Clover Park has not done a formal assessment of student 
performance gains. 

lutractloMl DtUvtiy 

The system can prviu* 2500 >i0urs of instniction at levels K.12. Many of the WICAT competencies are 
similar or identical to those of the Washington State Core Competencies ABE and CED curricula, but not the 
ESL curriculum. It seems particularly well-thought-out in the CED area, which is based on the 1988 CED. 
Although it is used with some ESL students they must be somewhat literate and able to under;»tand spoken 
English. The Clover Park VTI Lab Manager agrees that the ESL cuniculum is not as well developed as it 



Instruction is provided exclusively on the computer. The system docs not gericrate additional printed 
exercises for homework or refer the student to other media such as worid>ooks or videotapes. The instruc- 
tional modules make excellent use of the computer's branching and graphics capabilities and of its function 
keys. Generally lessons seem to be a reasonable length, although the length depends upon student perionn- 
ance. Many wrong answers and requests for help could lengthen the lesson. Eventually, if perfonnance is 
poor enough, the screen says "Stop — see your teacher'' and the student cannot proceed any farther. 

The instructor can determine the amount of student control. Lessons may be presented in a pre-assigned 
order, or the student may have complete control from a menu. 

Generally, program directions and instruction seem to be at approximately the same level of difficulty and 
are appropriate to the age and learning style of ABE and CED students. The Reading I curriculum has an 
audio component, which allows a student to hear as well as see directions and instructional material. (There 
are some concerns with the quality of the audio — see "Quality of System"). 

Th.ere is a variety of question and response types, sometimes choosing a multiple choice answer, sometimes 
underiining a word, sometimes typing in a response. In most uses, a student must respond and cannot 
advance simply by hitting the return key. The amount of practice seems sufficient. Most lessons allow a 
student 2 trials, then the answer is supplied. Hints are often given after an incorrect answer. Branching 
provides additional practice and review, but the system does not adjust the level or sequence of presentation 
based on student penomiance. The number of problems is limited if a student achieves mastery quickly or 
misses too many, but otherwise students can continue to request practice problems until they feel comfort- 
able with a skill. 

Guessing and random answers are discouraged, although not cc.npletely prevented. In some cases, after a 
student initially choose a wrong answer, ht is required to enter the correct answer from a model before 
being allowed to progress. 

Review questioiis are included, ^specially if a student h^ had difficulty with a particular concept. Queftions 
answered incorrectly arr not repeated, but the concept is usually reviewed. 

Progress is usually charted, sometimes within the lesson, so that a student is motivated to do well. There is 
often a graphic depiction of progress, such as checks after lessons mastered, the number correct shown on a 
b^r graph, etc. 

In the newer lessons, there is emphasis on the teaching of problem-solving and thinking skills, both in the 
type of problem presented and in the help given u'-on request or after an incorrect answer. The publisher 
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states that the system is "designed to provide students with the thinking skills they will find useful in the in- 
formation age.** 

Ftfdback 

After an answer, feedback is immediate and positive in tone. It neither threatens learners nor rewards 
incorrect responses. Some of the older modules, such as Spelling and Algebra, have feedback which is less 
varied and less helpful than the updated modules such as Reading. The older modules typically have a "No," 
or a **Thaf s right'" type of response. In the newer modules, the system will either explain why the choice was 
correct, or explain why it was wrong, give additional information, and encourage the student to try again. 
Feedback is content specific and related to a logical process of answer seeking. For example, when a 7 was 
entered instead of a 9, the computer responded "the 2 is not correct." 

Uaabfllty 

In order to implement the system, instructors need no computer knowledge; however, the system is complex 
enough that training would be imperative. Students also have no need of computer knowledge. A keyboard- 
ing lesson ttaches function keys, then on-screen commands and directions are user-friendly, making 
continued use of the system quite simple. M<^5t z^idtnt responses require one keystroke or one word. Both 
instructional and management systems are menu-diiven. 

An on-line calculator is available in appropriate activities. There are "repeat" and "back" keys which allow a 
student to repeat a frame or to back up several frames, or it is possible tr press the Escape key to leave the 
lesson. Generally, a student can either repeat or skip forward, but it depends upon what part of the lesson 
he's in. The "back" key is not always available, so in some cases, it would be necessary to escape the lesson 
and start again. A bookmark feature returns the student to the screen which was most recently completed 
during the previous lesson. 

Hints and on-line help are available with function keys; prompts at the bottom of each sc. een indicate what 
special keys are and when they're active However pressing the "information" key at some points moves 
student into what seems to be a totally unrelated help screen. At that point, the student can return to iue 
lesson by pressing the Escape key. 

The instructor can follow a menu to generate reports. The process is multi-step and somewhat complex, but 
if s possible to "save" a report format so that generating the same report a second time is much simpler. 
Reports are not available on-screen; they must be printed out Additionally, the management system may 
experience minor delays when many student stations are active. Updates to the system should not disturb 
student records, although the lab at Clover Park did experience a minor problem during a recent update. 
The lab manager feels that it was unusual and should not recur. 

NodlflibiUty 

It is simple to add WICAT modules or lessons to the management system, but it is not possible to add to or 
modify the content of a WICAT lesson. Using the student option menu, an instructor can easily change such 
parameters as rate and mastery level and can add a timer if it seems desirable to have a student complete a 
task within a preset time. Default settings are available also if the instructor does not wish to customize an 
assignment. 

WICAT now runs on an IBM I-class platform, so that the hardware can be used to run any IBM software. Al- 
though such software cannot be added into the WICAT system, it could be listed on the I-class menu. 

Naaalcincnt of Stadtnt Performance 

It is possible to assign objectives selectively for either a class or an individual student and is also possible to 
change the types of activities, the starting point in a sequence, and the amount of control over that sequence 
which the student will have. 

Within the context of WICAT curriculum materials, unneeded areas of the curnculum can be eliminated and 
other areas can be substituted. It is also reasonably simple to select certain objectives and "repackage'* them 
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as a separate course. Assessment within a lesson generally means drill until the skill is mastered or the 
student cannot get any correct If a student does not pass a test, he is given the option of repeating the 
lesson, but the system does not shift a student into a simpler area of the curriculum or an alternative 
presentation of the material. 

The instructor can change the scoring criteria for an objective or for a single student, altering the number of 
questions presented and the % correct for mastery. Student progress reports do include time on task, but 
mastety is equated only with % correct. 

The system will provide class rosters, individual progress reports, and class profiles. Since it's possible lo 
enter as many student ''fields" as desired, it would be possible to collect demographic data as well, but the 
system does not have a report format which would utilize it This would make it difficult to use the system to 
generate program statistics (such as number of students receiving public assistance, etc.) for proposals and 
reports. 

The system tracks and reports student time on each lesson as well as progress. It is very easy to add or delete 
students, to change or delete records, and to print records, since al! of these are menu choices. There is no 
record keeping which would need to be done manually. 

The system does not have a high degree of security. In order to access the management system, it is neces- 
saty only to know the code and to have access either to the management station or to an enhanced student 
station. Codes can easily be changed if necessary. It isn't likely that any software could be inadvertently 
erased or edited, since the "default" in every case is the option which is most protective of the system. 

Svpport/Strvkc 

The publisher does not impose any reporting requirements on the purchaser. There is technical suppcrt and 
service available locally and response to problems is immediate, dependent upon immediacy of need. A 
hotline is available, which is adequate for solving most problems. In a year of usage. Clover Park VTI has 
experienced one hardware problem. 

Staff training is provided with the initial purchase and ongoing staff development is provided as a part of the 
annual licensing fee. Clover Park has received only limited follow-up staff development and feels that may 
be because WICAT has only one trai ner available for the entire western region. 

WICAT has recently begun to hold occasional user's conferences. 

Sludettt Response 

The five students who were interviewed had never before participated in an individualized educational 
program and except for one had never used a computer in an educational program before. Three of the five 
felt that the computer instruction was preferable to more traditional methods of instruction, but one felt he 
would prefer to have more teacher involvement, and another felt that the computer gives so much help that 
she couldn't be sure she was actually learning materia! and would be able to apply it in other situations. 

All found the on-screen instruction easy to understand, but one mentioned that the instructions prior to 
each lesson are so long and detailed that they are confusing. The instructor confirmed that she frequently 
tells students to skip through the instructions. 

Students would like to be able to have worksheets to take home to reinforce what they are leammg. One 
student takes copious notes from the screen and then studies them at home. 

Features especially appreciated by the students include the bookmark feature which allows a student to begin 
where he last stopped, the immediate feedback, and the fun of using a computer. Problems mentioned 
included tlie barrier of keyboarding for someone who has never typed before, and a general feeling that in 
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order to benefit from the system if s necessary to be highly motivated — that otherwise it would be possible 
to ''slide through" exerting little effort They all fdt that this introduction to computer use would be of some 
value in later educational or occupational experiencci. 
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StcUoa 5: SUMMARY MATRIX 
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some of 
software 
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wiiLicii lur 
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sound 
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$$) 
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performance, 
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program data, 

student 
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profiles 


individual and 
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5000 student 
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Apple w/floppy 
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hard drive 
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CCC 


CCP 


ClatiWorhs 


PLATO 


WICAT 




yes 


yes 


yes 


yes 


yes 


StyrTnfaUag 


3*4 days initial, 
follow-up as 
needed, regular 
site visits 


6 days initial 
installation and 
training 
required, 2 
days follow-up 
recommended, 
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and user 
support group 


2 uays training 
and installation; 
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800# hotline, 
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technicians, 
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module, 
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modem 
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for systems 
operation, 
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800# hotline, 
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information is 
only included 
for general 
cstinution 
purposes. 
Systems vary 
widely and cost 
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negotiable, so 
none of these 
figures should 
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absolute.] 
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up, 1st yr. 
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recordcrs,5 
audiocassettes 
all needed 
print, AV, and 
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of 
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10-13% of 
purchase price, 
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APPENDIX 1 
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cfsnH sl/\nA Af\r\l^ 

didna-dione /\pp»c 


oarAii nogan 


deanie urDan League 


PPD 

CCr 


uary jonnson 


Qk«<fff VsIImi P/\1lArfA 

oKagii vciiey college 


PPD 

CCr 


nay naiey 


lacoma vommuriiky voiiege 


ll/lP AT 

WICA 1 


C^AnriSmiA A 11 Am 

diepnanie Alien 




stand-alone Commodore 64 


ucnnis noore 


ffdMiingion insuiuic oi iccnnotogy 


PPP 
ccc 


i\icndra Jonnson 


AD VP AM 




• 


oluc nounidin comm. college 


stana-aione Appie 


rain CI a Amsoerry 


cdfccr cnnancemenc cenier, Lauranae 


PPD 

CCr 


rtdry Ann mesncr 


PAnfv^l DrA<fAn P/\mm Pnll 

cenirai v/regon comm. coii. 


siana-aione nppie/iMdC 


L/idnne L^ean 


cencrai uregon incergovernmenidi councn, 


PPD 

CCr 


nereaiLn junge 


ncunionQ 






PhAffTiAl^A^A P/\nn9v« PaIIa^a 

cncmcKCvd comm. coiiege 


Ldc/josiens, cidssworKs, 


Lucy ludci/ondiQ 




necworKea Appies 




ciaciumds comm. college 


rainrinder 


Jim nesser 


Pla^eAff\ P/\fr.m P/\llArfA 

cidLsop coium. college 


sidna-aione Appie, ndc, ion 


UKOOIZ rvddpdi 


vommuniiy ocrvices consoruumf niudny 


PPP 
ccr 


PrAriTrmi Riiolt 

oregury oucii 


communiiy ocrvices consoirium cor'diiia 


PPD 

CCr 


joann uenoic 


J 00 council, neaiora 


^^^M/i Av%v%Ia 

stana-aione Appie 


x\en rrires 


iMdmdin Auuiw Luucduon rrogrdm 


LaucdLiori leLnnuiogy 


^iduuid ni^zuLU 


AidmdLii i^KC Ciiipiuy iiicrH irdining 


PPP 
CCr 


Pauls pAni^A 

rduid rerive 


illdUlULv 






L^ns Comrriunitv Coll^tff^ 


^t;ind.Alnnp Annl^ Mac IBM 

ouiiiU aiviiv ey^f^fi^f i tav^ twi i 


Leslie Rasor 


Linn>Benton Community College 


Apple instruction, 


Candice Johnson 




IBM record-keeping 




Mid-Willamette Jobs Council 


stand-alone IBM, NCR 


Dayna Bergeland 


Mt Hood Community College 


stand-alone Apple 


Brenda Button 


Oregon Coast Community College 


stand-alone Apple, IBM for 


Dawn DeWolf 




instruction and record-keepmg 


V. Tardacwcthcr 


Socrates Alternative Education 


stand-alone Apple and IBM 


Rick Focrtsch 


SW Oregon Community College 


stand-alone Apple, Mac, IBM 


Keith Hulsey 


Treasure Valley Community College 


CCP 


Art Clawson 


Umpqua Community College 


Education Technology 


Doris Leonard 


Career Enhancement Center, LaGrande 


CCP 


Mary Ann Micsner 
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■UriSH COLUMBU 

Cunosum College, Victoria 
Capitino College, North Vancouver 
Kmntlen College, Richmond 
Malupini College, Nanaimo 
Malaspina College, Powell River 
North bland College 
Northern Lights College 
Northwest Community College, 
Prince Rupert 

Northwest Community Coll., Terrace 
Okanagan College, Kelowna 
Okanagan College, Salmon Arm 
Selkirk College, Castelgar 
Selkirk College, Nelson Campus 
Vancouver Community College 



ERLC 



Systta 


Coatact PerfOtt 


r aLiiii nuci 


J. nosion 


Stand-alone Apple and IBM 


Robert Irvine 


stand-alone Apple and IBM 


M.E. Enns 


stand-alone IBM 


Paul Hurwitz 


stand-alone Apple and IBM 


Chris Tunstall 


stand-alone Aopple and IBM 


A.C. Lindsay 


record-keeping only 


Patricia Walker 


stand-alone Apple 


Marie Grinstrand 


stand-alone Apple 


Larry Boiingbroke 


stand-alone Mac 


Terry Buker 


stand-alone Apple 


Paul Grinder 


stand-alone Apple and IBM 


Cariton K. Scott 


Libcat (Apple and IBM) 


Marvin Work 


stand-alone IBM 


Cindy Onstad 
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APnDfDnti IVALUATIONlNSraUNBNT 



Nime of System: 

Publisher's Rep: 



Publisher 



Brief Description: 



ProCnm Where Observed: 
Contict Person: 



Length of Time System has been in place: 



Number of Students Served Monthly: 



Student user group<s): (ESU DE, ABE, etc.) 

Uses of system other than with basic skills students: 

What is relationship of system to the overall instructional program? 

Cnst Factors 

Estimated cost for work stations, file server, necessary staff training, set-up costs, initial licensing; 

Annual licensing fee: 

Is maintenance contracted or provided for an hourly charge? V/hat are the costs? What is included? 

What flexibility is there in the system design? Is it possible to buy only some of the components? 

Is it possible to customize the system by purchasing different modules? If so, what is the approximate cost per 



Is there a mode! of learning (such as learner-directed, teacher-directed, etc.) behind the system? What is it? 

What is the basis for the systeiH? (phonics, memorisation, problem solving, etc.) 

Are learner objectives stated clearly? Is purpose well defined? Do students know what is expected of them? 

What are the identified areas of instruction? What is the identified audience? (age, goal [such as high school 
credit GED prep, employment/life skills) 

Does the learning build on itself? (simple before complex, knowledge before synthesis) 



moduit? 



Are upgrades to the system available? If so, at what cost? How often? 



Is tile system expandable? cost per terminal exp^jision? 



A&stimnHQns about Lgamintf 
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Is the medium ind delivery appropriate for identified audience and content? Does the choice of medium vary 
with the student and/or topic? 

Would the system be usable with a student who dislikes computers? 

iJiabilitv 

How much computer knowledge is required of instructora to implement the system? of students? 

Do both instructional and management svstcms begin with a menu? 

Are directk>ns and on-scree commands and responses user-friendly? 

Is it easy (for the learner as well as the teacher) to retum to the menu at any time? 

Are there on-screen directions ano are they easy to access? 

Is it simple for a learner to review instructions and previous frames? 

Is there a 1)ookmark'' feature? 

!s help available at likely points of need? 

Wh?t level of typing skills is needed? (one stroke responses, single words, sentences and/or paragraphs) 

How easily can reports be generated by students and teachers? (test results, attendance reports, progress 
reports, etc) 

Do tne instructional and management systems run with minimal delays? 

Can the system be used easily with a variety of printers? 

Are student records affected (ex. deleted) when the svstem is updated? 

Modifiahiiitv 

Can new software be added to management and instructional systems? How easy is the process? 

Can an instructor easily change or add content to computer lessons? to supplemental matenals? (Are sup- 
plementary activities easily added/incorporated when students need more review or dnil?) 

Can an instructor easily change parameters such as rate or mastery level? 

Are default settings for these parameters available? 

Can the hardware be used for other purposes, i.e. word processing, programming, etc.? Are there modifica- 
tions required? 

Assg«mgnf/PUr»m»nt 

Is there a placement tool? If so, describe, including norming, quality, length, on-screen or paper/pencil, 
scannability. If not, describe the method for placing students in the system. 

Is the placement tool usable with all students? (ex: not with beginning readers, ESL. etc.) 

ERiC 
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k the student given information about time and effort needed to reach his/her goal (e.g., placement corre- 
lated to predicted GED scores.) 

Does this placement tool automatically lead into instruction or is it necessary for the instructor to enter as- 
signments? 

How long (teacher and student time) does it take to start a ii jw student? What type of orientation is avail- 
able for a new student? 

Describe the process of assessing progress, including cirteria to determine mastery. (How are tests admini- 
stered/graded? How soon are results available?)) 

Do the assessments measure the skills which have been taught? 

Are students learning? (Give data if available) 

R#gord^ M^natfgmgnt 

What records management features are available? What reports are available? (attendance, class rosters, in- 
dividual progress, class profiles, program statistics, student demographics, etc.) 

Can statistics be generated easily for reporting and fund raising? 

Who is responsible for tracking time and protrcss? (computer, instructor, student) 

Is it necessary to do any record keeping by hand? 

How difficult is it to add or delete students? 

How difficult is it to print records as often as wished? 

Can records be changed and/or deleted? 

Describe security measures in the system. Do students have access to any part of system other than instruc- 
tional? 

How destructible are the hardware and software components of the management system? 

ManAdi^mpnf of Student Pgrformance 
Is it possible to select only some of the objectives for a class or an individual student? 
Can unneeded areas of the curriculum be elimmated? Can other areas be added? 
Is it possible to select certain objectives and "repackage" them as a separate course? 
Can the instructor select mastery levels? How difficult is the process? 

Is nwvement through the curriculum based on assessment of mastery? Are ^ests based on level of student? 
level of material? How frequent are the assessments? 

What happens when a student does not pass a test? Is there '1)ranching'* which moves a student up. down, 
and sideways in the curriculum? 
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Does "mutenr reflect anything other than % correct? (such as error pattern or time elapsed between 
question and response?) 

How current is the material? (software as well as print and other media) 

Are updates available? How often and at what costs? 

I the sixe of print and clarity of screen image acceptable for adults? 

b ttie content accurate and free of '"bugs" as well as grammar, spelling, punctuation, and usage errors? 

Are graphics dear, easily interpreted, and appropriate for adults? 

b the audio clear? Can it be adjusted or turned off? 

bthe system documentation well done and easily used? 

How ^cnsh-prooT are the instructional and management systems? 

What is the breadth of the available curriculum? (What courses are available and how many hours of instruc- 
tion? academic? life skills? high school? GEO?) 

bthe curriculum appropriate for the Washington Core Competencies Curriculum? 

With what student levels would the system be useful? Does that match the published intended audience? 
Would it be useful with ESL students? 

What instructional and testing materials arc used? (computer? workbook? video cassette? tutoring? 
lecture? discussion?) 

What is the ratio of time spent on CAI to time spent with other media? 

Does the CAI portion of the curriculum take advantage of the computer's capabilities? (or is it simply drill 
and practice?) 

Were the materials (software, video, print, etc.) developed for this system or are they a collection of programs 
from various publishers? 

Will the system generate and print additional exercises for homework? Wtll it refer the student to other 
media for homework exercises? 

Are the units of instruction an appropriate length for a student's attentio'^ spai.? 

Does the system have a '^bookmark" feature so that the student can return to the same place at the next 
session? 

To %vhat extent is a student involved in the decision about what to study and/or emphasize? Is a student 
gh^n a menu of subject areas at each lesson? Can students alternate between math and reading, for ex- 
ample? 

Are the program directionand the instructional material at approximately the same level of difficulty? Are 
they appropriate to the age and culture and learning style of the intended audience? 
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Are any estimates given about time for a student attending r^ularly to complete a unit of the curriculum? If 
so, what m they? 

Can students truly work individually? Are **irofh\rf, infrequent attenders a problem? 

Does the student have control over the program; can he move backwards as well as forvvard and repeat when 
desired? 

Is there a variety of question types? What does the system do to "involve" the student in providing re- 
sponses? 

Is there sufficient exposure, opportunity for practice, and variety for a student to be able to master skills? Is 
there a limit to the number of problems a student can do in a section? 

Is there any means for preventing guessing or random answers? 

Is the number of trials allowed reasonable and appropriate? 

Are hints given after incorrect responses? 

Does the program use branching to automatically adjust difficulty levels or sequence ac cording to student 
performance? 

What provision is there for review? Is there an automatic system for incorporating review questions? 

Are questions which are answered incorrectly repeated later in the lesson or exercise? 

Is there transfer of learning? Do CAI skills transfer to pencil and papvr applications? Can the student apply 
learned skills to real-life situations? 



Are higher-order thinking skills taught 
What means is there to assist the student to feel he is progressing? 

Feedback 

What type of feedback is given for a correct answer? For an mcorrcct answer? 
Is the feedback positive and timely? 
Does feedback threaten learners or reward incorrect responses? 
Is feedback informative and corrective when appropriate? 
Is feedback varied? Is it ever boring or unnecessanly detailed? 

Support/Service 

Does the vendor/publisher impose any requirements such as reports or volume of use? 
Is there technical support and service available in this general area? 
Is there a hotline? 
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How much ttdiniol service have you required on the materiils, software, and hardware? 

b staff development included with system puvcnase? If not is it available? 

b there a mtworfc of usen which sen/e as a support group? Are conferences held for users? 

Staffing 

What level of staffing is necessary to operate the ^stem? 
Are cettifled teachers necessary or desired? 
Describe teacher duties/responsibilities. 
DefiCribe aide duties/responsibilities. 

Does the staff need a high level of expertise with computers? with the system? 
How much staff training is required? Is it adequate? 
b follow-up training available? What is the cost? 

How much paperwork/clerical time is needed? Is that time evenly distributed or concentrated in certain time 
periods? 

Fadlitigs Schi^diilintf 
What are the room/space requirements for the system? 
Is there a required attendance pattern or schedule for students? 
How many students can be served each day at one terminal? 
How long would a student be at the computer station at one sitting? 

Can the terminals be scheduled and used for other purposes such as programming or teaching word process- 
ing? 

Hardware 

What hardware is included in the basic system? color or monochrome monitors? 

Wtat is the base system storage capacity and operating system? 

b there expansion capability? 

Are there peripherals which are necessary/included? 

What type graphics and audio? 

Can students interact graphically? (pull-down menus, icons) 

4r 
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Have you been in an individualized educational program before? 

Have you used a computer in an educational program before? 

Do you think the computer instruction is better or worse than books and workbooks? 

Is it easy to understand the computer lessons? 

Can you do the lessons on paper after doing computer lessons? Which do you prefer? 
Do you ever feel frustrated? 

Do you enjoy using the program? What things do you like and dislike about it? 
Do you think this use of a computer will be useful in any other part of your life? 

nigral Qbsgrvations 

StrengthsAVeaknesses 

Cftmmgnt! 

Souce: 
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Intftfnted Uarnlng Syittms 
Chtddbt for Evtlaitlun 

Prograin Dtsigtt FKton: 

1. Stiffing needs: Clerical, Technical , Instructional 

2. Facilities 

3. Scheduling 

CostFictorv 

1. Cost ^fystem, including needed hardware and software and ongoing licensing or user fees. 

2. Posiibflity of leasing system rather than outright purchase. 

3. Possibiity of using already^owned hardware. 

4. Possibility of using hardware for otheer purposes, such as word processing or computerprogram- 
ming classes. 

5. lype of computers for ntanagement and student stations. 

6. Other hardware requirements (printer, modem, speech synthesizer, etc.) 

UsabUlty PactoTB 

1. Control over program: on-screen menus, ability to *l)ack up." 

2. Default settings for speed, mastery, etc, which can be either aacpted or modified. 

3. Level of computer expertise and typing skill needed for students and staff. 

4. Speed of response time in management and instructional software. 

5. General ease of use in generating reports, making assignments, modifying settings. 

6. Openness of ^tem: ability to add or rearrange curriculum. 

7. Support and Service provided 

Curriculum Factors 

1. Method of assessment and placement into system. 

2. Movement through curriculum (sequential branched, and/or spiraled) 

3. Clarity and relevance of leami ng objectives. 

4. Quality of curriculum materials, including instructional software. 

5. Currency of curriculum materials, including software. 

6. Variety of question and response types. 

7. Prevention of guessing or random answers 

8. Application of curriculum materials to life situations. 

9. Teaching as well as testingof higher order thinkingskills. 

10. Feedback that reinforces after correct responses and assists after mcorrect responses. 

11. Criteria for mastery 

12. Ability of student to see progress. 

ReconUnepliig FKton 

1. Amount of recordkeeping done manually vs. automatically. 

2. Amount of required recordkeeping 

3. Reports generated by system. 

4. Number of records which the system can store. 

5. Demographic and programm data which can be collected and reported 



